justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 294167.2@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID294167.2@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4127
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2024-11-18 16:22:50
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2024-11-18 18:14:55
From worker nodeHostnamewn1205251.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-18 16:23:37
Input filesfardet-hd:nu_dune10kt_1x2x6_1108_589_20230826T213357Z_gen_g4_detsim_hitreco__20240229T175945Z_reco2.root
JobscriptExit code0
Real time1h (6633s)
CPU time1h (6367s = 95%)
Outputting started2024-11-18 18:14:11
Output files
Finished2024-11-18 18:14:55
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/16/ce/nu_dune10kt_1x2x6_1108_589_20230826T213357Z_gen_g4_detsim_hitreco__20240229T175945Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Output 5: 0.0491987, 0.207514, 0.620216, 0.123072, 
Output 6: 0.999798, 0.000173672, 1.96663e-05, 8.89463e-06, 

Boundary wire vector sizes: 72, 94, 98
minwire 0: 2373
minwire 1: 21
minwire 2: 2741
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.122059, 
Output 1: 0.456195, 0.00211474, 0.0137569, 0.527933, 
Output 2: 0.897756, 0.00245876, 0.00118862, 0.0985963, 
Output 3: 0.592025, 0.361058, 0.0461437, 0.000773269, 
Output 4: 0.86572, 0.133763, 0.000460269, 5.72225e-05, 
Output 5: 0.999475, 0.000494232, 1.96762e-05, 1.07528e-05, 
Output 6: 0.988559, 0.0105748, 0.00066687, 0.000198886, 

%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:13:12 GMT run: 1108 subRun: 1 event: 58998
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:13:12 GMT run: 1108 subRun: 1 event: 58998
 Failed to find product with label caldata ... returning empty vector
%MSG
Boundary wire vector sizes: 651, 544, 601
minwire 0: 462
minwire 1: 1479
minwire 2: 555
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.235654, 
Output 1: 0.000643079, 0.000294003, 0.000750969, 0.998312, 
Output 2: 0.0839315, 0.207973, 0.595013, 0.113082, 
Output 3: 0.00713363, 0.0442255, 0.729417, 0.219224, 
Output 4: 0.975245, 0.0244843, 0.000262974, 7.40852e-06, 
Output 5: 0.000219433, 0.00472048, 0.991465, 0.00359525, 
Output 6: 0.998099, 0.0018315, 5.27283e-05, 1.63713e-05, 

%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:13:28 GMT run: 1108 subRun: 1 event: 58999
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:13:28 GMT run: 1108 subRun: 1 event: 58999
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:13:28 GMT run: 1108 subRun: 1 event: 58999
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:13:28 GMT run: 1108 subRun: 1 event: 58999
 Failed to find product with label caldata ... returning empty vector
%MSG
Boundary wire vector sizes: 1015, 1333, 1042
minwire 0: 2127
minwire 1: 133
minwire 2: 2091
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.00335438, 
Output 1: 0.719475, 0.00247036, 0.0371331, 0.240921, 
Output 2: 0.0247936, 0.142616, 0.832308, 0.000282275, 
Output 3: 0.404657, 0.587973, 0.00708136, 0.000288509, 
Output 4: 0.0780429, 0.215489, 0.429518, 0.27695, 
Output 5: 0.947192, 0.0521164, 0.000627303, 6.45931e-05, 
Output 6: 0.999598, 0.000366033, 2.59662e-05, 9.58629e-06, 

%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:13:53 GMT run: 1108 subRun: 1 event: 59000
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:13:53 GMT run: 1108 subRun: 1 event: 59000
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:14:01 GMT run: 1108 subRun: 1 event: 59000
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  18-Nov-2024 18:14:01 GMT run: 1108 subRun: 1 event: 59000
 Failed to find product with label caldata ... returning empty vector
%MSG
18-Nov-2024 18:14:09 GMT  Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/16/ce/nu_dune10kt_1x2x6_1108_589_20230826T213357Z_gen_g4_detsim_hitreco__20240229T175945Z_reco2.root"
18-Nov-2024 18:14:09 GMT  Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/16/ce/nu_dune10kt_1x2x6_1108_589_20230826T213357Z_gen_g4_detsim_hitreco__20240229T175945Z_reco2.root"

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 ccnuselection

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 6311.843971 Real = 6542.940113

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 4328.71 VmHWM = 2291.84

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2024-11-24 00:23:35 UTC       justIN version: 01.01.09