justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 295855.3@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID295855.3@justin-prod-sched01.dune.hep.ac.uk
Workflow ID4134
Stage ID1
User nameosbiston@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-19 04:56:42
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2024-11-20 00:25:06
From worker nodeHostnameccwcondor0586
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit106200 (29 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-19 20:36:40
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T221256Z_gen_001603_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time3h (13675s)
CPU time3h (12889s = 94%)
Outputting started2024-11-20 00:24:36
Output files
Finished2024-11-20 00:25:06
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-vd/5a/8f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T221256Z_gen_001603_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
Boundary wire vector sizes: 6810, 6726, 6717
minwire 0: 43
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.151259, 
Output 1: 0.0925114, 0.136178, 0.235942, 0.535369, 
Output 2: 0.0476684, 0.151962, 0.761259, 0.0391096, 
Output 3: 0.97154, 0.0158526, 0.00371031, 0.00889681, 
Output 4: 0.570608, 0.394147, 0.0315275, 0.00371742, 
Output 5: 0.380576, 0.334447, 0.201965, 0.0830115, 
Output 6: 0.0387019, 0.0937816, 0.0876768, 0.77984, 

Begin processing the 8th record. run: 3799 subRun: 0 event: 16028 at 20-Nov-2024 00:16:26 CET
Boundary wire vector sizes: 6720, 6711, 6620
minwire 0: 38
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.111026, 
Output 1: 0.102075, 0.175596, 0.155286, 0.567042, 
Output 2: 0.0670113, 0.147411, 0.759785, 0.0257929, 
Output 3: 0.976587, 0.0175354, 0.00136006, 0.00451719, 
Output 4: 0.653875, 0.319765, 0.022701, 0.00365896, 
Output 5: 0.389536, 0.369096, 0.162915, 0.0784521, 
Output 6: 0.0662586, 0.131372, 0.107013, 0.695356, 

Begin processing the 9th record. run: 3799 subRun: 0 event: 16029 at 20-Nov-2024 00:39:07 CET
Boundary wire vector sizes: 6532, 6467, 6696
minwire 0: 44
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2160, 2659
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0426959, 
Output 1: 0.0224129, 0.0639213, 0.0496575, 0.864008, 
Output 2: 0.049148, 0.229693, 0.696997, 0.0241611, 
Output 3: 0.915906, 0.0660756, 0.00887609, 0.00914238, 
Output 4: 0.939443, 0.0493779, 0.00824717, 0.00293243, 
Output 5: 0.412943, 0.452823, 0.117729, 0.0165053, 
Output 6: 0.00975324, 0.0406878, 0.0556535, 0.893905, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 16030 at 20-Nov-2024 01:01:16 CET
Boundary wire vector sizes: 6674, 6571, 6553
minwire 0: 73
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2248, 2747
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0866261, 
Output 1: 0.0687529, 0.0723064, 0.164185, 0.694755, 
Output 2: 0.0501431, 0.116659, 0.819585, 0.0136129, 
Output 3: 0.974303, 0.018186, 0.00189036, 0.00562006, 
Output 4: 0.379382, 0.525293, 0.0810654, 0.0142597, 
Output 5: 0.623268, 0.312338, 0.0495604, 0.0148327, 
Output 6: 0.0343347, 0.0836729, 0.0794547, 0.802538, 

20-Nov-2024 01:24:35 CET  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T221256Z_gen_001603_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-19T_203649Z.root"
20-Nov-2024 01:24:35 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-vd/5a/8f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T221256Z_gen_001603_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1308.89       1352.54       1394.62       1350.54       27.0685        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000855707   0.00125875    0.00277626    0.00102657    0.000550469      10     
reco:pandora:StandardPandora                            1307.07       1350.5        1392.83       1347.45       27.1188        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000270545    0.0045694     0.0427401    0.000317924    0.0127237       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000251239   0.000646168   0.00385389    0.000293173   0.00106946       10     
reco:pandoracalo:Calorimetry                          0.000223958    0.0254545     0.0362597     0.0301146     0.0124423       10     
reco:pandorapid:Chi2ParticleID                        6.2186e-05    0.000152557   0.000858722   7.5135e-05    0.000235528      10     
reco:cvnmap:CVNMapper                                  0.0594577     0.0721508     0.118466      0.0679262     0.015787        10     
reco:cvneva:CVNEvaluator                                0.87398       1.40867       3.51756       1.18748      0.714062        10     
reco:energyrecnumu:EnergyReco                          0.0579294     0.0697291     0.0983546     0.0664828     0.0110384       10     
reco:energyrecnue:EnergyReco                           0.0059397     0.0104198     0.0214283    0.00934336    0.00388286       10     
reco:energyrecnc:EnergyReco                           0.00648107     0.0105886     0.0224878    0.00923163    0.00413059       10     
[art]:TriggerResults:TriggerResultInserter             1.61e-05     3.3729e-05    0.000108373   2.2622e-05    2.6579e-05       10     
end_path:out1:RootOutput                               7.454e-06    1.16939e-05   3.4294e-05     9.613e-06    7.59225e-06      10     
end_path:out1:RootOutput(write)                         0.3153       0.434837      0.554444      0.431405      0.0569573       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19421.2 MB
  Peak resident set size usage (VmHWM): 2721.27 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2024-11-24 04:47:54 UTC       justIN version: 01.01.09