justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 104829.3@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID104829.3@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4146
Stage ID1
User nameosbiston@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-20 03:26:29
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2024-11-20 10:03:48
From worker nodeHostnameheplnc128.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-20 03:28:04
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193455Z_gen_000438_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time6h (23723s)
CPU time6h (22113s = 93%)
Outputting started2024-11-20 10:03:28
Output files
Finished2024-11-20 10:03:48
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-vd/56/3b/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193455Z_gen_000438_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
minwire 0: 33
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1675, 2174
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.101918, 
Output 1: 0.067924, 0.0694552, 0.171861, 0.69076, 
Output 2: 0.0481822, 0.178308, 0.750329, 0.0231814, 
Output 3: 0.951608, 0.0401738, 0.00409563, 0.0041222, 
Output 4: 0.422314, 0.498257, 0.0717039, 0.00772544, 
Output 5: 0.615193, 0.291288, 0.0745424, 0.0189776, 
Output 6: 0.019709, 0.070916, 0.0792133, 0.830162, 

Begin processing the 8th record. run: 3799 subRun: 0 event: 4378 at 20-Nov-2024 08:10:53 GMT
Boundary wire vector sizes: 6618, 6619, 6586
minwire 0: 75
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.111115, 
Output 1: 0.0512496, 0.0621481, 0.146492, 0.74011, 
Output 2: 0.0369378, 0.186908, 0.75627, 0.0198851, 
Output 3: 0.959021, 0.0263462, 0.00512849, 0.00950386, 
Output 4: 0.381923, 0.516027, 0.0916892, 0.0103606, 
Output 5: 0.503102, 0.398164, 0.0848778, 0.0138567, 
Output 6: 0.0122308, 0.0393391, 0.0564129, 0.892017, 

Begin processing the 9th record. run: 3799 subRun: 0 event: 4379 at 20-Nov-2024 08:48:51 GMT
Boundary wire vector sizes: 6420, 6409, 6603
minwire 0: 11
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1145, 1644
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.105582, 
Output 1: 0.185148, 0.0483265, 0.177041, 0.589485, 
Output 2: 0.063879, 0.185852, 0.723324, 0.0269453, 
Output 3: 0.940157, 0.0506569, 0.00496131, 0.00422495, 
Output 4: 0.515621, 0.423934, 0.0537811, 0.00666394, 
Output 5: 0.501783, 0.383443, 0.0937851, 0.0209892, 
Output 6: 0.0738473, 0.155205, 0.11485, 0.656097, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 4380 at 20-Nov-2024 09:26:39 GMT
Boundary wire vector sizes: 6867, 6863, 6770
minwire 0: 47
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 571, 1070
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0673181, 
Output 1: 0.0357504, 0.0545516, 0.0683857, 0.841312, 
Output 2: 0.0817891, 0.281216, 0.602926, 0.0340695, 
Output 3: 0.964632, 0.0275362, 0.00209393, 0.00573786, 
Output 4: 0.358578, 0.59077, 0.0477637, 0.0028879, 
Output 5: 0.838572, 0.134248, 0.0202246, 0.00695551, 
Output 6: 0.0127885, 0.0373727, 0.0541086, 0.89573, 

20-Nov-2024 10:03:27 GMT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193455Z_gen_000438_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-20T_032809Z.root"
20-Nov-2024 10:03:27 GMT  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-vd/56/3b/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193455Z_gen_000438_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2201.44       2355.43       2558.58       2302.8        126.731        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000958523   0.00698744     0.0334391    0.00128987     0.0102339       10     
reco:pandora:StandardPandora                            2198.6        2351.85       2555.22       2297.9        126.751        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000336619   0.00497657     0.0447095    0.000383585    0.0132515       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000363272   0.000945861    0.0048572    0.000424832   0.00133187       10     
reco:pandoracalo:Calorimetry                          0.00231119     0.0390621     0.051225      0.0416992     0.0135622       10     
reco:pandorapid:Chi2ParticleID                        7.9508e-05    0.000329343    0.0011638    0.00010776    0.000369145      10     
reco:cvnmap:CVNMapper                                  0.0854492     0.130196       0.31576      0.106709      0.0655287       10     
reco:cvneva:CVNEvaluator                                1.85737       2.56636       5.85364       2.16017       1.12541        10     
reco:energyrecnumu:EnergyReco                           0.12637      0.170406      0.228674      0.159864      0.0317731       10     
reco:energyrecnue:EnergyReco                           0.0124515     0.0134181     0.0157197     0.0130764    0.000996912      10     
reco:energyrecnc:EnergyReco                            0.0122623     0.0133439     0.0147842     0.0130877    0.000843189      10     
[art]:TriggerResults:TriggerResultInserter            1.8444e-05    3.94701e-05   0.000110332   3.0923e-05    2.52765e-05      10     
end_path:out1:RootOutput                              1.0276e-05    1.76787e-05   3.4764e-05    1.43805e-05   7.67191e-06      10     
end_path:out1:RootOutput(write)                        0.421016       0.62969      0.808331      0.624861      0.102455        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 10831 MB
  Peak resident set size usage (VmHWM): 2798.91 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2024-11-24 00:18:28 UTC       justIN version: 01.01.09