justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 104973.166@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID104973.166@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4154
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2024-11-20 10:43:51
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce03
Last heartbeat2024-11-20 12:11:57
From worker nodeHostnameccwcondor0484
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit106200 (29 hours)
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-20 11:04:03
Input filesfardet-hd:nu_dune10kt_1x2x6_1102_534_20230826T070155Z_gen_g4_detsim_hitreco__20240229T184311Z_reco2.root
JobscriptExit code0
Real time1h (4051s)
CPU time1h (4072s = 100%)
Outputting started2024-11-20 12:11:35
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/ccnuselection_nu_1/04154/1/nu_dune10kt_1x2x6_1102_534_20230826T070155Z_gen_g4_detsim_hitreco__20240229T184311Z_reco2_ccnutree.root
Finished2024-11-20 12:11:57
Saved logsjustin-logs:104973.166-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_91_02
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof

local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/baeee8261a966c7bf48ffdcd205bcc1e0b341bf3/localProducts_larsoft_v09_91_02_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/b6/88/nu_dune10kt_1x2x6_1102_534_20230826T070155Z_gen_g4_detsim_hitreco__20240229T184311Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
minwire 2: 2189
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2379, 2878
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0615681, 
Output 1: 0.0385834, 0.000254507, 0.0130078, 0.948154, 
Output 2: 0.318683, 0.263055, 0.146592, 0.271671, 
Output 3: 0.159313, 0.642949, 0.1883, 0.00943805, 
Output 4: 0.0769625, 0.880312, 0.0416917, 0.00103359, 
Output 5: 0.99807, 0.00177999, 0.000115337, 3.51529e-05, 
Output 6: 0.998324, 0.00139599, 0.000214156, 6.5524e-05, 

%MSG-w ShowerPCADirection:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 13:10:39 CET run: 1102 subRun: 1 event: 53499
0 spacepoints in shower, not calculating direction
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 13:10:39 CET run: 1102 subRun: 1 event: 53499
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 13:10:39 CET run: 1102 subRun: 1 event: 53499
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 13:10:39 CET run: 1102 subRun: 1 event: 53499
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 13:10:39 CET run: 1102 subRun: 1 event: 53499
Trying to add data product: ShowerPCA. This element does not exist in the element holder
%MSG
<ERROR>                          : 0-th variable of the event is NaN --> return MVA value -999, 
<ERROR>                          :  that's all I can do, please fix or remove this event.
<ERROR>                          : 0-th variable of the event is NaN --> return MVA value -999, 
<ERROR>                          :  that's all I can do, please fix or remove this event.
<ERROR>                          : 0-th variable of the event is NaN --> return MVA value -999, 
<ERROR>                          :  that's all I can do, please fix or remove this event.
<ERROR>                          : 0-th variable of the event is NaN --> return MVA value -999, 
<ERROR>                          :  that's all I can do, please fix or remove this event.
<ERROR>                          : 0-th variable of the event is NaN --> return MVA value -999, 
<ERROR>                          :  that's all I can do, please fix or remove this event.
Boundary wire vector sizes: 845, 781, 858
minwire 0: 804
minwire 1: 1550
minwire 2: 488
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.896015, 
Output 1: 0.000302535, 0.0540631, 0.0131628, 0.932472, 
Output 2: 0.0489308, 0.357904, 0.580027, 0.0131382, 
Output 3: 0.650344, 0.348716, 0.000685687, 0.000254287, 
Output 4: 0.659664, 0.33598, 0.0039447, 0.000411255, 
Output 5: 0.027284, 0.345725, 0.62277, 0.00422078, 
Output 6: 0.999185, 0.00069205, 9.64299e-05, 2.66727e-05, 

%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 13:10:57 CET run: 1102 subRun: 1 event: 53500
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 13:10:57 CET run: 1102 subRun: 1 event: 53500
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 13:10:57 CET run: 1102 subRun: 1 event: 53500
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShowerRedo@BeginModule  20-Nov-2024 13:10:57 CET run: 1102 subRun: 1 event: 53500
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
Boundary wire vector sizes: 1564, 1467, 1706
minwire 0: 1469
minwire 1: 32
minwire 2: 1713
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.128653, 
Output 1: 0.987367, 1.61118e-05, 0.0124905, 0.000126852, 
Output 2: 0.000197865, 0.0497206, 0.949375, 0.000706263, 
Output 3: 0.0033033, 0.0235454, 0.116511, 0.856641, 
Output 4: 0.965206, 0.0343444, 0.000442544, 6.79163e-06, 
Output 5: 0.000864654, 0.999064, 7.11682e-05, 5.4682e-07, 
Output 6: 0.99803, 0.00189735, 6.08122e-05, 1.20694e-05, 

20-Nov-2024 13:11:33 CET  Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/b6/88/nu_dune10kt_1x2x6_1102_534_20230826T070155Z_gen_g4_detsim_hitreco__20240229T184311Z_reco2.root"
20-Nov-2024 13:11:33 CET  Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/b6/88/nu_dune10kt_1x2x6_1102_534_20230826T070155Z_gen_g4_detsim_hitreco__20240229T184311Z_reco2.root"

TrigReport ---------- Event summary -------------
TrigReport Events total = 100 passed = 100 failed = 0

TrigReport ---------- Modules in End-path ----------
TrigReport        Run    Success      Error Name
TrigReport        100        100          0 ccnuselection

TimeReport ---------- Time summary [sec] -------
TimeReport CPU = 4029.233206 Real = 3997.851277

MemReport  ---------- Memory summary [base-10 MB] ------
MemReport  VmPeak = 4049.06 VmHWM = 2317.37

Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2024-11-23 12:20:20 UTC       justIN version: 01.01.09