justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 246487.50@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID246487.50@justin-prod-sched01.dune.hep.ac.uk
Workflow ID2901
Stage ID1
User nameismerio@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-08-24 00:26:58
SiteUK_Imperial
EntryDUNE_T2_UK_London_IC_ceprod00
Last heartbeat2024-08-24 03:41:46
From worker nodeHostnamewg28.grid.hep.ph.ic.ac.uk
cpuinfoIntel(R) Xeon(R) CPU X5650 @ 2.67GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-08-24 00:27:35
Input filesfardet-hd:nue_dune10kt_1x2x6_1118_990_20230827T223911Z_gen_g4_detsim_hitreco__20240221T044017Z_reco2.root
JobscriptExit code0
Real time3h (11584s)
CPU time3h (11189s = 96%)
Outputting started2024-08-24 03:40:52
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ismerio/02901/1/Validation_nue_dune10kt_1x2x6_1118_990_20230827T223911Z_gen_g4_detsim_hitreco__20240221T044017Z_reco2.root
Finished2024-08-24 03:41:46
Saved logsjustin-logs:246487.50-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

cegrid.org/products/common/db/
CETPKG_INSTALL=/exp/dune/app/users/ismerio/env-testreco/localProducts_larsoft_v09_91_02_e26_prof

local product directory is /exp/dune/app/users/ismerio/env-testreco/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
WARNING: Unable to open file /exp/dune/app/users/ismerio/env-testreco/localProducts_larsoft_v09_91_02_e26_prof.
---------------------------------------------------------
Art output set to nue_dune10kt_1x2x6_1118_990_20230827T223911Z_gen_g4_detsim_hitreco__20240221T044017Z_reco2_reco_data_2024-08-24T_002742Z.root
Contents of /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/c5fe39c11c378ea944c1ce1c47f8dbacb94669f9:
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
localProducts_larsoft_v09_91_02_e26_prof
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
lar exit code 0
=== Start last 100 lines of lar log file ===
  57
  71
Track tracks are:
  2
  4
  13
  16
  30
  38
  44
  54
  62
  67
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  24-Aug-2024 04:40:20 BST run: 1118 subRun: 1 event: 99100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  24-Aug-2024 04:40:20 BST run: 1118 subRun: 1 event: 99100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  24-Aug-2024 04:40:20 BST run: 1118 subRun: 1 event: 99100
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  24-Aug-2024 04:40:20 BST run: 1118 subRun: 1 event: 99100
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 1185, 1113, 1028
minwire 0: 2208
minwire 1: 159
minwire 2: 2264
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.128897, 
Output 1: 0.00534014, 0.938434, 0.0424665, 0.0137591, 
Output 2: 0.108434, 0.0404652, 0.00052186, 0.850579, 
Output 3: 0.101901, 0.848258, 0.0487176, 0.00112363, 
Output 4: 0.990878, 0.00911816, 3.62277e-06, 1.18345e-07, 
Output 5: 0.99729, 0.00250034, 6.57886e-05, 0.000144034, 
Output 6: 0.0104403, 0.0251486, 0.0493691, 0.915042, 

677
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2505!
24-Aug-2024 04:40:35 BST  Closed output file "nue_dune10kt_1x2x6_1118_990_20230827T223911Z_gen_g4_detsim_hitreco__20240221T044017Z_reco2_reco_data_2024-08-24T_002742Z.root"
24-Aug-2024 04:40:35 BST  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/a0/e0/nue_dune10kt_1x2x6_1118_990_20230827T223911Z_gen_g4_detsim_hitreco__20240221T044017Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.639597       113.041       1231.3        56.2497       198.787        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0293034     0.101858      0.490602      0.0787022     0.0676536       100    
reco:linecluster:LineCluster                           0.0675419      2.02432       65.1107      0.227433       8.22924        100    
reco:trajcluster:TrajCluster                           0.025649       8.0634        168.299       0.48682       25.3199        100    
reco:pandora:StandardPandora                           0.0486853      7.34102       99.5182       3.33731       13.7613        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000189081     0.02067      0.180939     0.00859461     0.036137        100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000230716    0.024238      0.136285      0.0117714     0.0297547       100    
reco:pandoracalo:Calorimetry                          0.000147401    0.117699       2.65574      0.0173486     0.361171        100    
reco:pandorapid:Chi2ParticleID                        5.7967e-05    0.000308808   0.00209487    0.000241413   0.000256386      100    
reco:trkshowersplit:TrackShowerHits                   0.000141607    0.0792717      1.19825      0.0112259     0.210837        100    
reco:pmtrack:PMAlgTrackMaker                          0.00172595      11.5955       179.764       3.14374       28.0064        100    
reco:pmtrackcalo:Calorimetry                          0.000109062    0.0566685     0.784397      0.012262      0.138133        100    
reco:pmtrackpid:Chi2ParticleID                        4.7215e-05    0.00104952    0.00874795    0.000700856   0.00132726       100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000291071     11.376        306.729       2.90775       33.4133        100    
reco:pmtrajfitcalo:Calorimetry                        0.000110227    0.123356       2.62354      0.0198217     0.369801        100    
reco:pmtrajfitpid:Chi2ParticleID                       4.791e-05    0.00041298    0.00181181    0.000361589   0.000292729      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00152474      14.6839       187.039       4.01063       33.3173        100    
reco:pmtracktccalo:Calorimetry                        0.000108709    0.0323574     0.319616     0.00990011     0.0608407       100    
reco:pmtracktcpid:Chi2ParticleID                      4.7023e-05    0.000874046   0.00544626    0.000632058   0.000894025      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000248424     6.23106       61.6979       2.74498       11.3621        100    
reco:pmtrajfittccalo:Calorimetry                      0.000110908    0.0691936     0.982414      0.0145075     0.163984        100    
reco:pmtrajfittcpid:Chi2ParticleID                    4.8246e-05    0.00112593     0.0061154    0.000759401   0.00125138       100    
reco:blurredcluster:BlurredClustering                 0.000247425     2.00749       9.32642       1.50701       1.82244        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000150146     40.9677       276.135       25.5487       53.7844        100    
reco:emshower:EMShower                                0.000219642    0.0861644     0.439323      0.0522664     0.0967711       100    
reco:cvnmap:CVNMapper                                 3.1784e-05     0.0530048      0.14191      0.0592472     0.0234134       100    
reco:cvneva:CVNEvaluator                              2.5159e-05      3.51936       9.78205       3.47803       1.42569        100    
reco:energyrecnumu:EnergyReco                         0.000677544    0.175049       2.85751      0.0276267      0.42387        100    
reco:energyrecnue:EnergyReco                          0.000243556    0.0067803     0.0603615     0.0026915     0.0118165       100    
reco:energyrecnc:EnergyReco                           0.00250746    0.00987003     0.0655426     0.0056702     0.0126622       100    
reco:energyrecnumurange:EnergyReco                    0.000330171    0.166324       2.83864      0.0186468     0.422429        100    
reco:energyrecnumumcs:EnergyReco                      0.00251387     0.168347       2.83551      0.022657      0.421238        100    
reco:opdec:Deconvolution                               0.0599253     0.414386      0.652026      0.430545      0.128392        100    
reco:ophitspe:OpHitFinderDeco                          0.221734       2.77356       5.33679       2.83438      0.990497        100    
reco:opflash:OpFlashFinder                            0.000209329   0.00246574     0.0288613    0.00219987     0.0027696       100    
reco:rns:RandomNumberSaver                            2.1875e-05    3.67359e-05   0.000325224   3.4269e-05    2.98363e-05      100    
[art]:TriggerResults:TriggerResultInserter            1.5008e-05    1.89883e-05   5.1279e-05    1.7984e-05    4.67253e-06      100    
end_path:out1:RootOutput                               3.144e-06    6.28381e-06   2.0527e-05    6.4265e-06    2.67445e-06      100    
end_path:out1:RootOutput(write)                        0.0128094     0.744398       1.85068       0.71805      0.333976        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 7639.68 MB
  Peak resident set size usage (VmHWM): 2757.88 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_nue_dune10kt_1x2x6_1118_990_20230827T223911Z_gen_g4_detsim_hitreco__20240221T044017Z_reco2.root
all-input-dids.txt
debugprod.log
jobscript.log
justin-processed-pfns.txt
nue_dune10kt_1x2x6_1118_990_20230827T223911Z_gen_g4_detsim_hitreco__20240221T044017Z_reco2_reco_2024-08-24T_002742Z.log
nue_dune10kt_1x2x6_1118_990_20230827T223911Z_gen_g4_detsim_hitreco__20240221T044017Z_reco2_reco_data_2024-08-24T_002742Z.root
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
localProducts_larsoft_v09_91_02_e26_prof
justIN time: 2024-11-17 08:43:39 UTC       justIN version: 01.01.09