justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 99473.9@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID99473.9@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4122
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2024-11-16 17:32:15
SiteNL_SURFsara
EntryDUNE_SurfSARA_arc03
Last heartbeat2024-11-16 19:16:57
From worker nodeHostnamewn-ha-28.gina.surfsara.nl
cpuinfoAMD EPYC 7551 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit129600 (36 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-16 17:33:15
Input filesfardet-hd:nu_dune10kt_1x2x6_1062_541_20230823T121119Z_gen_g4_detsim_hitreco__20240229T184335Z_reco2.root
JobscriptExit code0
Real time1h (6202s)
CPU time1h (5949s = 95%)
Outputting started2024-11-16 19:16:37
Output files
Finished2024-11-16 19:16:57
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/b1/8a/nu_dune10kt_1x2x6_1062_541_20230823T121119Z_gen_g4_detsim_hitreco__20240229T184335Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Output 2: 0.323577, 0.219563, 0.191306, 0.265554, 
Output 3: 0.193205, 0.60461, 0.196704, 0.00547979, 
Output 4: 0.160512, 0.228853, 0.407638, 0.202997, 
Output 5: 0.320422, 0.67902, 0.000538143, 2.04871e-05, 
Output 6: 0.997806, 0.00201632, 0.000129533, 4.86065e-05, 

Boundary wire vector sizes: 2149, 2614, 2460
minwire 0: 1087
minwire 1: 455
minwire 2: 719
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0696474, 
Output 1: 0.770676, 0.000208643, 0.144503, 0.0846124, 
Output 2: 0.00302824, 0.0123807, 0.984429, 0.000161852, 
Output 3: 0.702575, 0.291834, 0.00458201, 0.00100966, 
Output 4: 0.00281882, 0.0281256, 0.206124, 0.762931, 
Output 5: 0.993228, 0.00648242, 0.000259608, 2.99204e-05, 
Output 6: 0.997472, 0.00239253, 0.000100726, 3.4823e-05, 

%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 20:15:38 CET run: 1062 subRun: 1 event: 54197
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 20:15:38 CET run: 1062 subRun: 1 event: 54197
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 20:15:54 CET run: 1062 subRun: 1 event: 54197
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 20:15:54 CET run: 1062 subRun: 1 event: 54197
 Failed to find product with label caldata ... returning empty vector
%MSG
Boundary wire vector sizes: 841, 794, 971
minwire 0: 1264
minwire 1: 775
minwire 2: 1290
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.619537, 
Output 1: 0.9903, 4.71591e-05, 0.00764913, 0.00200411, 
Output 2: 0.0204112, 0.477325, 0.2714, 0.230864, 
Output 3: 0.00303024, 0.861341, 0.135558, 7.07901e-05, 
Output 4: 0.380549, 0.618489, 0.000946265, 1.59664e-05, 
Output 5: 0.999943, 5.05922e-05, 4.66274e-06, 1.65194e-06, 
Output 6: 0.998029, 0.00174027, 0.000202224, 2.87972e-05, 

Boundary wire vector sizes: 240, 210, 182
minwire 0: 259
minwire 1: 1922
minwire 2: 91
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2373, 2872
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.531602, 
Output 1: 0.000401591, 0.000681638, 0.000417589, 0.998499, 
Output 2: 0.0441473, 0.633994, 0.256191, 0.0656682, 
Output 3: 0.536003, 0.455747, 0.00788065, 0.000369259, 
Output 4: 0.990667, 0.00923725, 9.25369e-05, 2.83561e-06, 
Output 5: 0.00808568, 0.985512, 0.00636241, 4.02455e-05, 
Output 6: 0.996644, 0.00303902, 0.000273673, 4.35085e-05, 

Boundary wire vector sizes: 325, 398, 354
minwire 0: 1173
minwire 1: 1488
minwire 2: 1220
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.253666, 
Output 1: 3.57064e-05, 9.00817e-05, 0.0019694, 0.997905, 
Output 2: 0.298291, 0.334707, 0.143732, 0.223271, 
Output 3: 0.0333185, 0.157534, 0.806633, 0.00251503, 
Output 4: 0.917691, 0.08149, 0.000792271, 2.66398e-05, 
Output 5: 0.0145707, 0.978591, 0.00670617, 0.000132524, 
Output 6: 0.999579, 0.000373673, 3.63547e-05, 1.05427e-05, 

16-Nov-2024 20:16:36 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/b1/8a/nu_dune10kt_1x2x6_1062_541_20230823T121119Z_gen_g4_detsim_hitreco__20240229T184335Z_reco2.root"
16-Nov-2024 20:16:36 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/b1/8a/nu_dune10kt_1x2x6_1062_541_20230823T121119Z_gen_g4_detsim_hitreco__20240229T184335Z_reco2.root"

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 ccnuselection

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 5898.216873 Real = 6138.668636

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3798.48 VmHWM = 2080.27

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2024-11-17 03:56:35 UTC       justIN version: 01.01.09