justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 101321.9@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID101321.9@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4127
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2024-11-18 14:48:01
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep207
Last heartbeat2024-11-18 16:57:47
From worker nodeHostnameheplnc108.pp.rl.ac.uk
cpuinfoIntel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-18 14:49:31
Input filesfardet-hd:nu_dune10kt_1x2x6_1062_399_20230823T120928Z_gen_g4_detsim_hitreco__20240227T205849Z_reco2.root
JobscriptExit code0
Real time2h (7515s)
CPU time1h (7156s = 95%)
Outputting started2024-11-18 16:54:46
Output files
Finished2024-11-18 16:57:47
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/ae/8d/nu_dune10kt_1x2x6_1062_399_20230823T120928Z_gen_g4_detsim_hitreco__20240227T205849Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===

%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  18-Nov-2024 16:44:02 GMT run: 1062 subRun: 1 event: 39995
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  18-Nov-2024 16:44:02 GMT run: 1062 subRun: 1 event: 39995
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 2636, 2236, 1739
minwire 0: 224
minwire 1: 1563
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0971329, 
Output 1: 0.171069, 0.683855, 0.041655, 0.103421, 
Output 2: 0.00815746, 0.100781, 0.89088, 0.000181589, 
Output 3: 0.768161, 0.221409, 0.00943828, 0.000991625, 
Output 4: 0.186969, 0.507071, 0.278322, 0.0276381, 
Output 5: 0.269714, 0.682985, 0.0436563, 0.00364488, 
Output 6: 0.999552, 0.000375964, 4.89147e-05, 2.32059e-05, 

Boundary wire vector sizes: 1376, 1431, 1669
minwire 0: 1404
minwire 1: 291
minwire 2: 1294
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0428859, 
Output 1: 0.99561, 6.20337e-05, 0.00414932, 0.0001787, 
Output 2: 0.0235432, 0.0751512, 0.0132607, 0.888045, 
Output 3: 0.00337072, 0.0191106, 0.976334, 0.00118437, 
Output 4: 0.99902, 0.00097679, 2.98516e-06, 2.84717e-07, 
Output 5: 0.999986, 1.16483e-05, 1.77198e-06, 1.03838e-06, 
Output 6: 0.995779, 0.00393983, 0.000240895, 4.00284e-05, 

Boundary wire vector sizes: 998, 1170, 1245
minwire 0: 1266
minwire 1: 1701
minwire 2: 645
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0393782, 
Output 1: 0.99085, 2.43421e-05, 0.00867631, 0.000449755, 
Output 2: 0.0381135, 0.629993, 0.218293, 0.113601, 
Output 3: 0.00461487, 0.37726, 0.610386, 0.00773855, 
Output 4: 0.996073, 0.00389224, 3.34766e-05, 9.45412e-07, 
Output 5: 0.0210296, 0.978833, 0.00012448, 1.25811e-05, 
Output 6: 0.997229, 0.00257196, 0.000167163, 3.22011e-05, 

%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  18-Nov-2024 16:49:24 GMT run: 1062 subRun: 1 event: 39999
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  18-Nov-2024 16:49:24 GMT run: 1062 subRun: 1 event: 39999
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  18-Nov-2024 16:49:24 GMT run: 1062 subRun: 1 event: 39999
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  18-Nov-2024 16:49:24 GMT run: 1062 subRun: 1 event: 39999
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 2555, 2429, 2590
minwire 0: 1272
minwire 1: 481
minwire 2: 1192
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.13906, 
Output 1: 0.747892, 0.00607494, 0.125071, 0.120962, 
Output 2: 0.0110025, 0.0205613, 0.9637, 0.00473609, 
Output 3: 0.0515165, 0.540459, 0.331997, 0.0760278, 
Output 4: 0.0298882, 0.101593, 0.219528, 0.648991, 
Output 5: 0.212434, 0.619411, 0.15902, 0.00913483, 
Output 6: 0.999289, 0.000621906, 5.30773e-05, 3.59807e-05, 

18-Nov-2024 16:54:41 GMT  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/ae/8d/nu_dune10kt_1x2x6_1062_399_20230823T120928Z_gen_g4_detsim_hitreco__20240227T205849Z_reco2.root"
18-Nov-2024 16:54:41 GMT  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/ae/8d/nu_dune10kt_1x2x6_1062_399_20230823T120928Z_gen_g4_detsim_hitreco__20240227T205849Z_reco2.root"

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 ccnuselection

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 7098.618464 Real = 7399.712494

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 4102.59 VmHWM = 2518.25

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2024-11-24 04:15:50 UTC       justIN version: 01.01.09