justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 105927.50@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID105927.50@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4198
Stage ID1
User nameosbiston@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-21 13:26:22
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce508
Last heartbeat2024-11-21 20:50:45
From worker nodeHostnameb9g37p8598.cern.ch
cpuinfoIntel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-21 13:27:28
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193640Z_gen_000802_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time7h (26573s)
CPU time6h (25037s = 94%)
Outputting started2024-11-21 20:50:23
Output files
Finished2024-11-21 20:50:45
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/fardet-vd/e0/c2/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193640Z_gen_000802_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
minwire 0: 83
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2000, 2499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.113237, 
Output 1: 0.0553026, 0.0844085, 0.166434, 0.693855, 
Output 2: 0.0377687, 0.203143, 0.737863, 0.021225, 
Output 3: 0.954468, 0.0330573, 0.00522166, 0.00725319, 
Output 4: 0.272116, 0.615762, 0.103748, 0.00837367, 
Output 5: 0.591841, 0.321582, 0.0722905, 0.0142865, 
Output 6: 0.0181756, 0.0576695, 0.072656, 0.851499, 

Begin processing the 8th record. run: 3799 subRun: 0 event: 8018 at 21-Nov-2024 19:48:07 CET
Boundary wire vector sizes: 6928, 6869, 6804
minwire 0: 30
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 744, 1243
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.105512, 
Output 1: 0.0843633, 0.0915095, 0.0920672, 0.73206, 
Output 2: 0.0514293, 0.23489, 0.681319, 0.0323608, 
Output 3: 0.963749, 0.013243, 0.00398934, 0.0190189, 
Output 4: 0.802443, 0.173292, 0.020161, 0.00410348, 
Output 5: 0.32348, 0.512151, 0.140039, 0.0243298, 
Output 6: 0.0286711, 0.0577308, 0.0688738, 0.844724, 

Begin processing the 9th record. run: 3799 subRun: 0 event: 8019 at 21-Nov-2024 20:25:58 CET
Boundary wire vector sizes: 6807, 6785, 6757
minwire 0: 22
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.110487, 
Output 1: 0.0681388, 0.0528735, 0.271684, 0.607303, 
Output 2: 0.107066, 0.128471, 0.740735, 0.023728, 
Output 3: 0.904031, 0.0888763, 0.0048573, 0.00223523, 
Output 4: 0.24389, 0.631968, 0.109443, 0.0146992, 
Output 5: 0.56043, 0.366272, 0.058515, 0.0147829, 
Output 6: 0.100664, 0.203719, 0.125835, 0.569782, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 8020 at 21-Nov-2024 21:09:29 CET
Boundary wire vector sizes: 6686, 6609, 6561
minwire 0: 23
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1594, 2093
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.183686, 
Output 1: 0.0995967, 0.13042, 0.268288, 0.501696, 
Output 2: 0.0259068, 0.108524, 0.849836, 0.0157339, 
Output 3: 0.973709, 0.0215434, 0.00210958, 0.00263771, 
Output 4: 0.285769, 0.60736, 0.0967734, 0.0100973, 
Output 5: 0.251349, 0.335337, 0.300482, 0.112833, 
Output 6: 0.0647609, 0.153179, 0.124294, 0.657767, 

21-Nov-2024 21:50:19 CET  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193640Z_gen_000802_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-21T_132737Z.root"
21-Nov-2024 21:50:19 CET  Closed input file "root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/fardet-vd/e0/c2/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193640Z_gen_000802_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2264.11       2602.15       2985.56       2554.31       232.994        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00090908    0.00240246     0.0108243    0.00123443    0.00293886       10     
reco:pandora:StandardPandora                            2260.94       2598.64       2982.2        2551.27       232.414        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000388613   0.00474649     0.0375694    0.00108851     0.0109648       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000409081   0.00112396    0.00599421    0.000506801   0.00163849       10     
reco:pandoracalo:Calorimetry                          0.00241569     0.0395773     0.0543913     0.0423892     0.0149347       10     
reco:pandorapid:Chi2ParticleID                        0.000105093   0.000226222   0.00115208    0.000118244   0.000309117      10     
reco:cvnmap:CVNMapper                                  0.0964328      0.15187      0.248014       0.12526      0.0523804       10     
reco:cvneva:CVNEvaluator                                1.41744       2.40696        5.72         2.14714       1.14126        10     
reco:energyrecnumu:EnergyReco                          0.100678      0.152306      0.214244       0.14899      0.0326107       10     
reco:energyrecnue:EnergyReco                           0.0105106     0.0136313     0.0156874     0.0139235    0.00160932       10     
reco:energyrecnc:EnergyReco                            0.0105796     0.0140197     0.0186171     0.0139063    0.00208469       10     
[art]:TriggerResults:TriggerResultInserter            2.0983e-05    3.9577e-05    0.00012158    2.88975e-05   2.84232e-05      10     
end_path:out1:RootOutput                               8.104e-06    2.40854e-05   8.4376e-05    1.44655e-05   2.32751e-05      10     
end_path:out1:RootOutput(write)                        0.498029      0.719508      0.988908      0.707533      0.113071        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9763.44 MB
  Peak resident set size usage (VmHWM): 2602.18 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2024-11-21 22:56:02 UTC       justIN version: 01.01.09