Jobsub ID 297429.109@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	297429.109@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4146
Stage ID	1
User name	osbiston@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-19 21:21:38
Site	FR_CCIN2P3
Entry	DUNE_FR_CCIN2P3_cccondorce04
Last heartbeat	2024-11-20 00:50:24
From worker node	Hostname	ccwcondor0397
	cpuinfo	AMD EPYC 9334 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	106200 (29 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-19 21:22:51
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193327Z_gen_000297_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	3h (12428s)
	CPU time	3h (12003s = 96%)
Outputting started	2024-11-20 00:50:00
Output files
Finished	2024-11-20 00:50:24
List job events Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-vd/e1/92/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193327Z_gen_000297_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
minwire 1: 0
minwire 2: 2
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 792, 1291
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.125271, 
Output 1: 0.0953098, 0.192981, 0.10206, 0.609649, 
Output 2: 0.0703429, 0.193685, 0.70741, 0.0285619, 
Output 3: 0.984935, 0.00769775, 0.000853973, 0.00651311, 
Output 4: 0.8375, 0.1493, 0.0107267, 0.00247424, 
Output 5: 0.304172, 0.482606, 0.161315, 0.0519068, 
Output 6: 0.0335991, 0.0721847, 0.0800839, 0.814132, 

Begin processing the 8th record. run: 3799 subRun: 0 event: 2968 at 20-Nov-2024 00:47:01 CET
Boundary wire vector sizes: 6760, 6688, 6628
minwire 0: 60
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1026, 1525
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.153094, 
Output 1: 0.0654863, 0.0492148, 0.134579, 0.75072, 
Output 2: 0.0640336, 0.210154, 0.689534, 0.0362783, 
Output 3: 0.976098, 0.0157088, 0.00235851, 0.00583432, 
Output 4: 0.360893, 0.566636, 0.0665867, 0.00588453, 
Output 5: 0.472752, 0.384676, 0.115035, 0.0275367, 
Output 6: 0.0240259, 0.0608881, 0.0741874, 0.840899, 

Begin processing the 9th record. run: 3799 subRun: 0 event: 2969 at 20-Nov-2024 01:07:06 CET
Boundary wire vector sizes: 6787, 6646, 6629
minwire 0: 36
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 451, 950
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.115457, 
Output 1: 0.435726, 0.0607368, 0.23464, 0.268897, 
Output 2: 0.040862, 0.0584189, 0.892646, 0.00807284, 
Output 3: 0.968024, 0.0287933, 0.00113541, 0.00204763, 
Output 4: 0.54604, 0.413399, 0.0327891, 0.00777106, 
Output 5: 0.219699, 0.462814, 0.216006, 0.101481, 
Output 6: 0.204693, 0.28793, 0.128374, 0.379003, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 2970 at 20-Nov-2024 01:27:56 CET
Boundary wire vector sizes: 6794, 6758, 6796
minwire 0: 12
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 694, 1193
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0452106, 
Output 1: 0.0725958, 0.0951898, 0.0651548, 0.76706, 
Output 2: 0.0426063, 0.16206, 0.78115, 0.0141828, 
Output 3: 0.922651, 0.0705997, 0.003764, 0.00298564, 
Output 4: 0.863787, 0.12035, 0.0126985, 0.00316367, 
Output 5: 0.165983, 0.526663, 0.25706, 0.0502938, 
Output 6: 0.07881, 0.305605, 0.173763, 0.441823, 

20-Nov-2024 01:49:59 CET  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193327Z_gen_000297_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-19T_212255Z.root"
20-Nov-2024 01:49:59 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-vd/e1/92/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T193327Z_gen_000297_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1125.81       1232.85       1320.17       1233.64       53.4174        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000578597   0.00380436     0.0185416    0.000816752   0.00579501       10     
reco:pandora:StandardPandora                            1124.55       1231.26       1318.8        1232.28       53.2849        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.00023096    0.00410877     0.0369537    0.000253203    0.0109625       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000224901   0.000897558   0.00676434    0.000248667   0.00195566       10     
reco:pandoracalo:Calorimetry                          0.00166309     0.0230008     0.0289111     0.0241225    0.00735486       10     
reco:pandorapid:Chi2ParticleID                        5.5283e-05    0.000140969   0.00077209    6.22895e-05   0.000211952      10     
reco:cvnmap:CVNMapper                                  0.0526044     0.0610807     0.0710436     0.0603199     0.0050771       10     
reco:cvneva:CVNEvaluator                               0.740378       1.04888       2.84265      0.841376      0.604186        10     
reco:energyrecnumu:EnergyReco                          0.0551425     0.0754833     0.0981995     0.0724966     0.0160394       10     
reco:energyrecnue:EnergyReco                          0.00782378     0.0092258     0.0174048    0.00826685    0.00276132       10     
reco:energyrecnc:EnergyReco                           0.00804335     0.0096321     0.0210687    0.00824437    0.00384053       10     
[art]:TriggerResults:TriggerResultInserter            1.3711e-05    2.52533e-05   9.5495e-05     1.641e-05    2.37761e-05      10     
end_path:out1:RootOutput                               9.054e-06    1.21524e-05   3.0066e-05     1.015e-05    6.02449e-06      10     
end_path:out1:RootOutput(write)                        0.312369      0.357918      0.472432      0.344294      0.0429357       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21681.1 MB
  Peak resident set size usage (VmHWM): 2858.9 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0