justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 298166.0@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID298166.0@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4183
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2024-11-20 22:24:32
SiteUS_Colorado
EntryCMSHTPC_T3_US_Colorado_heposg01-colorado
Last heartbeat2024-11-21 01:03:50
From worker nodeHostnamelnxfarm197.colorado.edu
cpuinfoIntel(R) Core(TM) i7 CPU 950 @ 3.07GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-20 22:26:26
Input filesfardet-hd:nu_dune10kt_1x2x6_1421_29_20230827T093316Z_gen_g4_detsim_hitreco__20240223T223557Z_reco2.root
JobscriptExit code0
Real time2h (9427s)
CPU time2h (9787s = 103%)
Outputting started2024-11-21 01:03:35
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/ccnuselection_nu_3/04183/1/nu_dune10kt_1x2x6_1421_29_20230827T093316Z_gen_g4_detsim_hitreco__20240223T223557Z_reco2_ccnutree.root
Finished2024-11-21 01:03:50
Saved logsjustin-logs:298166.0-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/5b/72/nu_dune10kt_1x2x6_1421_29_20230827T093316Z_gen_g4_detsim_hitreco__20240223T223557Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
%MSG
%MSG-e DUNEAna:  dunereco/FDSelections/CCNuSelection:ccnuselection@BeginModule  20-Nov-2024 18:02:58 MST run: 1421 subRun: 1 event: 2996
 Failed to find product with label caldata ... returning empty vector
%MSG
Boundary wire vector sizes: 240, 269, 282
minwire 0: 395
minwire 1: 1901
minwire 2: 323
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.752838, 
Output 1: 9.4494e-06, 6.28927e-06, 4.56483e-06, 0.99998, 
Output 2: 0.463355, 0.342442, 0.143855, 0.0503487, 
Output 3: 0.999971, 2.90383e-05, 8.21495e-08, 7.51958e-08, 
Output 4: 0.999991, 9.03016e-06, 4.14735e-08, 3.17402e-09, 
Output 5: 6.26987e-05, 0.999908, 2.86739e-05, 6.22204e-07, 
Output 6: 0.998227, 0.00153939, 0.000222328, 1.11291e-05, 

Boundary wire vector sizes: 483, 362, 394
minwire 0: 1525
minwire 1: 1599
minwire 2: 1127
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2375, 2874
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.102829, 
Output 1: 5.84326e-05, 0.000101653, 0.035574, 0.964266, 
Output 2: 0.904662, 0.0186465, 0.0019731, 0.074718, 
Output 3: 0.0127144, 0.986813, 0.000459515, 1.28203e-05, 
Output 4: 0.00875619, 0.991068, 0.000173097, 2.47524e-06, 
Output 5: 0.655692, 0.343256, 0.000997488, 5.35934e-05, 
Output 6: 0.99934, 0.000582273, 6.59251e-05, 1.19779e-05, 

Boundary wire vector sizes: 608, 716, 689
minwire 0: 1740
minwire 1: 655
minwire 2: 1897
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0149239, 
Output 1: 0.898373, 0.000459886, 0.0320779, 0.0690889, 
Output 2: 0.00695493, 0.190627, 0.802212, 0.000205741, 
Output 3: 0.999103, 0.000896749, 1.69045e-07, 2.68858e-08, 
Output 4: 0.010067, 0.977657, 0.0122681, 8.17375e-06, 
Output 5: 0.00364076, 0.989632, 0.00667417, 5.35267e-05, 
Output 6: 0.999854, 0.000128771, 1.60354e-05, 1.4972e-06, 

%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 18:03:28 MST run: 1421 subRun: 1 event: 3000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 18:03:28 MST run: 1421 subRun: 1 event: 3000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 18:03:28 MST run: 1421 subRun: 1 event: 3000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 18:03:28 MST run: 1421 subRun: 1 event: 3000
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 456, 554, 455
minwire 0: 424
minwire 1: 1222
minwire 2: 49
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.57508, 
Output 1: 0.0023598, 0.00104602, 0.0346571, 0.961937, 
Output 2: 0.29442, 0.284058, 0.116555, 0.304967, 
Output 3: 0.306276, 0.645307, 0.0460107, 0.00240588, 
Output 4: 0.0610565, 0.32342, 0.589723, 0.025801, 
Output 5: 0.04885, 0.948042, 0.00302782, 8.0385e-05, 
Output 6: 0.984217, 0.014196, 0.0012228, 0.000364054, 

20-Nov-2024 18:03:32 MST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/5b/72/nu_dune10kt_1x2x6_1421_29_20230827T093316Z_gen_g4_detsim_hitreco__20240223T223557Z_reco2.root"
20-Nov-2024 18:03:32 MST  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/5b/72/nu_dune10kt_1x2x6_1421_29_20230827T093316Z_gen_g4_detsim_hitreco__20240223T223557Z_reco2.root"

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 ccnuselection

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 9742.197186 Real = 9376.310167

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 3991.98 VmHWM = 2473.43

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2024-11-21 14:55:53 UTC       justIN version: 01.01.09