justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 100296.4@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID100296.4@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4122
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2024-11-16 22:45:22
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2024-11-17 00:52:01
From worker nodeHostnamewn-ha-43.gina.surfsara.nl
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-16 22:46:56
Input filesfardet-hd:nu_dune10kt_1x2x6_1098_567_20230826T013109Z_gen_g4_detsim_hitreco__20240229T184625Z_reco2.root
JobscriptExit code0
Real time2h (7487s)
CPU time2h (7605s = 101%)
Outputting started2024-11-17 00:51:43
Output files
Finished2024-11-17 00:52:01
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/83/64/nu_dune10kt_1x2x6_1098_567_20230826T013109Z_gen_g4_detsim_hitreco__20240229T184625Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
minwire 0: 162
minwire 1: 1597
minwire 2: 76
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.168938, 
Output 1: 0.00413478, 0.0171098, 0.267836, 0.710919, 
Output 2: 0.00611635, 0.0272909, 0.964745, 0.00184768, 
Output 3: 0.858795, 0.134165, 0.00566089, 0.0013789, 
Output 4: 0.0640761, 0.291618, 0.520365, 0.123941, 
Output 5: 0.0359922, 0.14879, 0.445511, 0.369707, 
Output 6: 0.997666, 0.00198871, 0.000227545, 0.000117602, 

Boundary wire vector sizes: 1359, 942, 1344
minwire 0: 320
minwire 1: 68
minwire 2: 85
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.866616, 
Output 1: 0.976659, 0.000193908, 0.0222271, 0.00091999, 
Output 2: 0.84153, 0.0329841, 0.0146622, 0.110824, 
Output 3: 0.989055, 0.00855811, 0.00136227, 0.00102452, 
Output 4: 0.996832, 0.00306076, 8.32201e-05, 2.43256e-05, 
Output 5: 0.994323, 0.0053832, 0.000228199, 6.59311e-05, 
Output 6: 0.991676, 0.00679947, 0.0011573, 0.00036705, 

ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  17-Nov-2024 01:49:47 CET run: 1098 subRun: 1 event: 56799
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  17-Nov-2024 01:49:47 CET run: 1098 subRun: 1 event: 56799
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  17-Nov-2024 01:49:47 CET run: 1098 subRun: 1 event: 56799
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  17-Nov-2024 01:49:47 CET run: 1098 subRun: 1 event: 56799
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 247, 192, 231
minwire 0: 432
minwire 1: 2529
minwire 2: 20
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.174731, 
Output 1: 0.00212236, 2.66632e-05, 0.000627343, 0.997224, 
Output 2: 0.127681, 0.665215, 0.0726351, 0.134469, 
Output 3: 0.0306831, 0.270266, 0.642325, 0.0567261, 
Output 4: 0.31031, 0.662656, 0.0266199, 0.000414318, 
Output 5: 0.998662, 0.00130119, 3.19472e-05, 4.38178e-06, 
Output 6: 0.999557, 0.000383988, 4.38169e-05, 1.53676e-05, 

%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  17-Nov-2024 01:49:55 CET run: 1098 subRun: 1 event: 56800
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  17-Nov-2024 01:49:55 CET run: 1098 subRun: 1 event: 56800
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 2905, 2350, 3074
minwire 0: 103
minwire 1: 448
minwire 2: 42
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.338052, 
Output 1: 0.978699, 2.9169e-05, 0.0209961, 0.000275416, 
Output 2: 0.027944, 0.689786, 0.22937, 0.0529001, 
Output 3: 0.000369249, 0.00151643, 0.00344538, 0.994669, 
Output 4: 0.988237, 0.0116541, 0.000106125, 2.94329e-06, 
Output 5: 0.999923, 6.92033e-05, 5.86173e-06, 2.28846e-06, 
Output 6: 0.994674, 0.00518285, 0.000109667, 3.33729e-05, 

17-Nov-2024 01:51:42 CET  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/83/64/nu_dune10kt_1x2x6_1098_567_20230826T013109Z_gen_g4_detsim_hitreco__20240229T184625Z_reco2.root"
17-Nov-2024 01:51:42 CET  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/83/64/nu_dune10kt_1x2x6_1098_567_20230826T013109Z_gen_g4_detsim_hitreco__20240229T184625Z_reco2.root"

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 ccnuselection

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 7561.590558 Real = 7433.075409

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 4214.24 VmHWM = 2500.89

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2024-11-17 06:18:17 UTC       justIN version: 01.01.09