justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 297432.140@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID297432.140@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4146
Stage ID1
User nameosbiston@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-19 21:22:43
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2024-11-20 02:14:24
From worker nodeHostnameheplnc166.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-19 21:24:00
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T225947Z_gen_001272_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time4h (17398s)
CPU time4h (16592s = 95%)
Outputting started2024-11-20 02:13:58
Output files
Finished2024-11-20 02:14:24
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-vd/7d/07/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T225947Z_gen_001272_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
Boundary wire vector sizes: 6664, 6595, 6776
minwire 0: 68
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0903475, 
Output 1: 0.252078, 0.104152, 0.187929, 0.455841, 
Output 2: 0.0487507, 0.0782908, 0.864399, 0.0085593, 
Output 3: 0.958894, 0.0375473, 0.0011635, 0.00239494, 
Output 4: 0.443684, 0.499183, 0.0475186, 0.00961446, 
Output 5: 0.282305, 0.443316, 0.184592, 0.0897872, 
Output 6: 0.199786, 0.276762, 0.13787, 0.385581, 

Begin processing the 8th record. run: 3799 subRun: 0 event: 12718 at 20-Nov-2024 00:48:11 GMT
Boundary wire vector sizes: 6695, 6690, 6682
minwire 0: 11
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 704, 1203
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0429539, 
Output 1: 0.0343982, 0.0466911, 0.0767825, 0.842128, 
Output 2: 0.0772038, 0.264347, 0.628826, 0.0296234, 
Output 3: 0.869255, 0.108798, 0.0133802, 0.00856666, 
Output 4: 0.540088, 0.36956, 0.0786846, 0.0116674, 
Output 5: 0.8224, 0.158196, 0.0161488, 0.00325538, 
Output 6: 0.030318, 0.105195, 0.0983799, 0.766108, 

Begin processing the 9th record. run: 3799 subRun: 0 event: 12719 at 20-Nov-2024 01:16:50 GMT
Boundary wire vector sizes: 6944, 6895, 6734
minwire 0: 59
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.123891, 
Output 1: 0.0574269, 0.0432602, 0.0885698, 0.810743, 
Output 2: 0.148306, 0.140403, 0.693458, 0.0178337, 
Output 3: 0.991457, 0.00704246, 0.000142715, 0.00135807, 
Output 4: 0.0956562, 0.867471, 0.0349578, 0.00191468, 
Output 5: 0.448746, 0.459549, 0.064549, 0.0271557, 
Output 6: 0.0515649, 0.124117, 0.115253, 0.709065, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 12720 at 20-Nov-2024 01:44:59 GMT
Boundary wire vector sizes: 6453, 6402, 6446
minwire 0: 17
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 528, 1027
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.120882, 
Output 1: 0.0783123, 0.264701, 0.155663, 0.501323, 
Output 2: 0.0496578, 0.215989, 0.705127, 0.0292253, 
Output 3: 0.966697, 0.023909, 0.00309125, 0.00630289, 
Output 4: 0.666236, 0.295589, 0.0335218, 0.00465414, 
Output 5: 0.49378, 0.342112, 0.124235, 0.0398723, 
Output 6: 0.0433854, 0.0967473, 0.0948667, 0.765001, 

20-Nov-2024 02:13:57 GMT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T225947Z_gen_001272_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-19T_212406Z.root"
20-Nov-2024 02:13:57 GMT  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-vd/7d/07/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T225947Z_gen_001272_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1594.36       1724.09       1786.3        1734.13       55.6459        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00089558    0.00567681     0.0250679    0.00128319    0.00817449       10     
reco:pandora:StandardPandora                            1591.89       1721.41       1783.72       1731.77       55.4665        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00030515    0.00423227     0.0338598    0.000452375   0.00992452       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000304077   0.000739514   0.00410127    0.000372895   0.00112119       10     
reco:pandoracalo:Calorimetry                          0.000289781    0.0249334     0.0405507     0.0304725     0.0132327       10     
reco:pandorapid:Chi2ParticleID                        7.2495e-05    0.000170282   0.000871075   9.1976e-05    0.00023402       10     
reco:cvnmap:CVNMapper                                  0.0562373     0.0640696     0.0726847     0.0648066    0.00504828       10     
reco:cvneva:CVNEvaluator                                1.45736       1.82972       3.68659       1.63659      0.622927        10     
reco:energyrecnumu:EnergyReco                          0.121682      0.157295      0.245155      0.144322      0.0364173       10     
reco:energyrecnue:EnergyReco                           0.0124038     0.0129355     0.0135281     0.0128049    0.000403835      10     
reco:energyrecnc:EnergyReco                            0.0123318     0.0129204     0.0134381     0.0129549    0.00042248       10     
[art]:TriggerResults:TriggerResultInserter            1.8805e-05    3.68759e-05   0.000155721   2.5217e-05    3.97012e-05      10     
end_path:out1:RootOutput                              1.2714e-05    2.15811e-05   5.1696e-05    1.4737e-05    1.48773e-05      10     
end_path:out1:RootOutput(write)                        0.440607      0.564407      0.640314      0.570963      0.046995        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 38748 MB
  Peak resident set size usage (VmHWM): 2721.4 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2024-11-21 16:44:03 UTC       justIN version: 01.01.09