justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 99885.11@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID99885.11@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4118
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2024-11-16 20:25:59
SiteUS_FNAL-T1
EntryCMSHTPC_T1_US_FNAL_condce_opp1_whole
Last heartbeat2024-11-16 23:23:32
From worker nodeHostnamedunegli-36812-0-cmswn2270.fnal.gov
cpuinfoIntel(R) Xeon(R) CPU E5-2670 v3 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-16 20:55:06
Input filesfardet-hd:nutau_dune10kt_1x2x6_1110_277_20230827T092359Z_gen_g4_detsim_hitreco__20240220T184329Z_reco2.root
JobscriptExit code0
Real time2h (8894s)
CPU time2h (9438s = 106%)
Outputting started2024-11-16 23:23:21
Output files
Finished2024-11-16 23:23:32
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/c3/f5/nutau_dune10kt_1x2x6_1110_277_20230827T092359Z_gen_g4_detsim_hitreco__20240220T184329Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Output 5: 0.999931, 6.10894e-05, 5.95407e-06, 1.54996e-06, 
Output 6: 0.997113, 0.00266338, 0.000179532, 4.38986e-05, 

ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Boundary wire vector sizes: 269, 383, 355
minwire 0: 1344
minwire 1: 1517
minwire 2: 1028
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.322731, 
Output 1: 0.00668825, 0.00119542, 0.0199273, 0.972189, 
Output 2: 0.396995, 0.202079, 0.13359, 0.267336, 
Output 3: 0.979518, 0.019385, 0.000783841, 0.000313205, 
Output 4: 0.00497713, 0.963086, 0.0317378, 0.00019875, 
Output 5: 0.999041, 0.000848912, 8.9938e-05, 2.05627e-05, 
Output 6: 0.991649, 0.00678748, 0.00129607, 0.000267688, 

%MSG-w LArPandora:  StandardPandora:pandoraRedo@BeginModule  16-Nov-2024 23:22:35 UTC run: 1110 subRun: 1 event: 27799
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-w LArPandora:  StandardPandora:pandoraRedo@BeginModule  16-Nov-2024 23:22:35 UTC run: 1110 subRun: 1 event: 27799
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-w LArPandora:  StandardPandora:pandoraRedo@BeginModule  16-Nov-2024 23:22:35 UTC run: 1110 subRun: 1 event: 27799
CreatePandoraMCParticles - found an unknown process
%MSG
%MSG-w LArPandora:  StandardPandora:pandoraRedo@BeginModule  16-Nov-2024 23:22:35 UTC run: 1110 subRun: 1 event: 27799
CreatePandoraMCParticles - found an unknown process
%MSG
Boundary wire vector sizes: 1809, 1730, 1700
minwire 0: 340
minwire 1: 769
minwire 2: 181
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.23156, 
Output 1: 0.134205, 0.0575714, 0.402765, 0.405458, 
Output 2: 0.0526667, 0.038741, 0.889345, 0.0192475, 
Output 3: 0.956056, 0.0427187, 0.000667268, 0.000558288, 
Output 4: 0.081331, 0.540751, 0.341116, 0.0368015, 
Output 5: 0.93239, 0.0666154, 0.000910977, 8.33567e-05, 
Output 6: 0.999161, 0.000697062, 0.000104565, 3.70966e-05, 

%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 23:22:59 UTC run: 1110 subRun: 1 event: 27799
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 23:22:59 UTC run: 1110 subRun: 1 event: 27799
 Failed to find product with label caldata ... returning empty vector
%MSG
Boundary wire vector sizes: 379, 320, 387
minwire 0: 650
minwire 1: 1441
minwire 2: 731
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.868795, 
Output 1: 0.542677, 0.00388961, 0.0107999, 0.442633, 
Output 2: 0.624346, 0.135745, 0.0139278, 0.225981, 
Output 3: 0.997757, 0.00204947, 0.000137723, 5.63121e-05, 
Output 4: 0.450377, 0.548079, 0.00142376, 0.000119364, 
Output 5: 0.995375, 0.00450131, 9.58616e-05, 2.7382e-05, 
Output 6: 0.981391, 0.0160132, 0.00219646, 0.000399266, 

%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 23:23:20 UTC run: 1110 subRun: 1 event: 27800
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 23:23:20 UTC run: 1110 subRun: 1 event: 27800
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 23:23:20 UTC run: 1110 subRun: 1 event: 27800
 Failed to find product with label caldata ... returning empty vector
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  16-Nov-2024 23:23:20 UTC run: 1110 subRun: 1 event: 27800
 Failed to find product with label caldata ... returning empty vector
%MSG
16-Nov-2024 23:23:20 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/c3/f5/nutau_dune10kt_1x2x6_1110_277_20230827T092359Z_gen_g4_detsim_hitreco__20240220T184329Z_reco2.root"
16-Nov-2024 23:23:20 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/c3/f5/nutau_dune10kt_1x2x6_1110_277_20230827T092359Z_gen_g4_detsim_hitreco__20240220T184329Z_reco2.root"

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 ccnuselection

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 9385.480574 Real = 8838.137685

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 4115.86 VmHWM = 2409.33

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2024-11-17 06:54:45 UTC       justIN version: 01.01.09