justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 101176.9@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID101176.9@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4127
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2024-11-18 12:58:32
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2024-11-18 14:49:15
From worker nodeHostnameheplnc155.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-18 13:00:13
Input filesfardet-hd:nu_dune10kt_1x2x6_1410_549_20230826T123355Z_gen_g4_detsim_hitreco__20240229T182034Z_reco2.root
JobscriptExit code0
Real time1h (6520s)
CPU time2h (7585s = 116%)
Outputting started2024-11-18 14:48:53
Output files
Finished2024-11-18 14:49:15
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/55/ad/nu_dune10kt_1x2x6_1410_549_20230826T123355Z_gen_g4_detsim_hitreco__20240229T182034Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Output 5: 0.999924, 6.27527e-05, 1.24323e-05, 1.10204e-06, 
Output 6: 0.999121, 0.000804225, 5.49315e-05, 1.99165e-05, 

%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 14:47:03 GMT run: 1410 subRun: 1 event: 54996
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 14:47:03 GMT run: 1410 subRun: 1 event: 54996
 Failed to find product with label caldata ... returning empty vector
%MSG
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Boundary wire vector sizes: 110, 66, 62
minwire 0: 1948
minwire 1: 612
minwire 2: 1530
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.717124, 
Output 1: 0.0585888, 0.000231175, 0.00541211, 0.935768, 
Output 2: 0.411896, 0.0850789, 0.0205084, 0.482517, 
Output 3: 0.939565, 0.0565177, 0.00347775, 0.000439734, 
Output 4: 0.114539, 0.882039, 0.00322434, 0.000197516, 
Output 5: 0.999508, 0.00045204, 2.50078e-05, 1.51459e-05, 
Output 6: 0.988945, 0.00939514, 0.00132067, 0.000338949, 

ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  18-Nov-2024 14:47:13 GMT run: 1410 subRun: 1 event: 54998
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  18-Nov-2024 14:47:13 GMT run: 1410 subRun: 1 event: 54998
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 2402, 1987, 2581
minwire 0: 657
minwire 1: 557
minwire 2: 415
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.479105, 
Output 1: 0.979874, 5.64617e-05, 0.0184674, 0.00160188, 
Output 2: 0.059319, 0.607018, 0.201399, 0.132264, 
Output 3: 0.0302196, 0.331847, 0.568144, 0.0697894, 
Output 4: 0.504907, 0.472326, 0.022026, 0.000741476, 
Output 5: 0.999178, 0.000793373, 2.37426e-05, 5.37511e-06, 
Output 6: 0.997808, 0.00197167, 0.00016255, 5.76176e-05, 

Boundary wire vector sizes: 1193, 1268, 1373
minwire 0: 2001
minwire 1: 316
minwire 2: 2113
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.00679919, 
Output 1: 0.996071, 9.77304e-06, 0.00372187, 0.00019717, 
Output 2: 0.000358373, 0.0175953, 0.982018, 2.85832e-05, 
Output 3: 0.999369, 0.000627553, 2.94605e-06, 1.19994e-07, 
Output 4: 0.000236717, 0.999458, 0.0003052, 3.99224e-07, 
Output 5: 0.000448737, 0.999382, 0.000168058, 1.10051e-06, 
Output 6: 0.998202, 0.00166174, 0.000128225, 7.9062e-06, 

Boundary wire vector sizes: 418, 339, 443
minwire 0: 1288
minwire 1: 1007
minwire 2: 1417
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.77419, 
Output 1: 0.98861, 0.000175951, 0.0102741, 0.000939958, 
Output 2: 0.845185, 0.00649542, 0.00189867, 0.146421, 
Output 3: 0.999359, 0.000603534, 3.20301e-05, 5.52326e-06, 
Output 4: 0.999161, 0.000826443, 9.48046e-06, 2.81143e-06, 
Output 5: 0.999939, 4.93166e-05, 7.90605e-06, 3.7577e-06, 
Output 6: 0.941338, 0.0513293, 0.00636965, 0.000963036, 

18-Nov-2024 14:48:51 GMT  Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/55/ad/nu_dune10kt_1x2x6_1410_549_20230826T123355Z_gen_g4_detsim_hitreco__20240229T182034Z_reco2.root"
18-Nov-2024 14:48:51 GMT  Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/55/ad/nu_dune10kt_1x2x6_1410_549_20230826T123355Z_gen_g4_detsim_hitreco__20240229T182034Z_reco2.root"

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 ccnuselection

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 7543.806499 Real = 6470.989006

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 5011.26 VmHWM = 2382.75

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2024-11-23 23:11:15 UTC       justIN version: 01.01.09