Jobsub ID 295473.0@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	295473.0@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4134
Stage ID	1
User name	osbiston@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-19 02:06:03
Site	FR_CCIN2P3
Entry	DUNE_FR_CCIN2P3_cccondorce01
Last heartbeat	2024-11-19 16:11:19
From worker node	Hostname	ccwcondor0540
	cpuinfo	AMD EPYC 7302 16-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	106200 (29 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-19 10:49:30
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T232234Z_gen_002557_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	5h (19284s)
	CPU time	5h (18400s = 95%)
Outputting started	2024-11-19 16:10:54
Output files
Finished	2024-11-19 16:11:19
List job events Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-vd/42/ab/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T232234Z_gen_002557_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
minwire 1: 0
minwire 2: 2
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1304, 1803
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0687549, 
Output 1: 0.0241204, 0.0727084, 0.0784791, 0.824692, 
Output 2: 0.0663851, 0.298771, 0.605216, 0.0296277, 
Output 3: 0.930916, 0.0561581, 0.00642922, 0.00649678, 
Output 4: 0.472916, 0.463573, 0.0586694, 0.00484129, 
Output 5: 0.574697, 0.368139, 0.0503543, 0.00681014, 
Output 6: 0.0181124, 0.0681201, 0.0813338, 0.832434, 

Begin processing the 8th record. run: 3799 subRun: 0 event: 25568 at 19-Nov-2024 15:36:24 CET
Boundary wire vector sizes: 6572, 6525, 6525
minwire 0: 4
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1703, 2202
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.11199, 
Output 1: 0.536729, 0.0451738, 0.213932, 0.204166, 
Output 2: 0.0346393, 0.0660805, 0.888819, 0.0104615, 
Output 3: 0.952893, 0.041238, 0.00255548, 0.00331375, 
Output 4: 0.652515, 0.318769, 0.0233682, 0.0053483, 
Output 5: 0.221592, 0.412791, 0.2491, 0.116518, 
Output 6: 0.178302, 0.281271, 0.129972, 0.410455, 

Begin processing the 9th record. run: 3799 subRun: 0 event: 25569 at 19-Nov-2024 16:08:05 CET
Boundary wire vector sizes: 6808, 6725, 6620
minwire 0: 51
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1130, 1629
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.107418, 
Output 1: 0.0357201, 0.148709, 0.146734, 0.668836, 
Output 2: 0.0386956, 0.247185, 0.685641, 0.0284784, 
Output 3: 0.93711, 0.0416057, 0.00929738, 0.0119868, 
Output 4: 0.418151, 0.497581, 0.0774687, 0.00679872, 
Output 5: 0.51097, 0.369547, 0.101341, 0.0181421, 
Output 6: 0.0179637, 0.0555182, 0.0724253, 0.854093, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 25570 at 19-Nov-2024 16:39:35 CET
Boundary wire vector sizes: 6866, 6839, 6691
minwire 0: 43
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 318, 817
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.111818, 
Output 1: 0.169214, 0.0777675, 0.161949, 0.591069, 
Output 2: 0.0352038, 0.0854876, 0.86649, 0.0128186, 
Output 3: 0.982474, 0.0107455, 0.00119916, 0.00558141, 
Output 4: 0.629589, 0.334492, 0.0294115, 0.00650752, 
Output 5: 0.185264, 0.457032, 0.262557, 0.0951471, 
Output 6: 0.0872312, 0.156938, 0.111219, 0.644611, 

19-Nov-2024 17:10:53 CET  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T232234Z_gen_002557_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-19T_104934Z.root"
19-Nov-2024 17:10:53 CET  Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-vd/42/ab/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T232234Z_gen_002557_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1867.02       1916.36       1967.58       1905.71       36.2133        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000974741   0.00332621     0.0201224    0.00130924    0.00561696       10     
reco:pandora:StandardPandora                            1865.17       1914.27       1965.68       1902.43       36.2275        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000278475   0.00499114     0.0405833    0.00113823     0.0118825       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000271031   0.000907906   0.00623309    0.000300386   0.00177575       10     
reco:pandoracalo:Calorimetry                          0.00229065     0.0272793     0.0365511     0.0279637    0.00900593       10     
reco:pandorapid:Chi2ParticleID                        7.0163e-05    0.000179313   0.00103659    7.8068e-05    0.000286351      10     
reco:cvnmap:CVNMapper                                  0.0574497     0.0749771     0.136621      0.0670589     0.0233343       10     
reco:cvneva:CVNEvaluator                               0.932267       1.46688       3.48075       1.27234      0.688151        10     
reco:energyrecnumu:EnergyReco                          0.0509109     0.0865424     0.148415      0.0789886     0.0261939       10     
reco:energyrecnue:EnergyReco                          0.00693032     0.0104286     0.0113611     0.0107729    0.00120272       10     
reco:energyrecnc:EnergyReco                           0.00941798     0.0106982     0.0115012     0.0107482    0.000570348      10     
[art]:TriggerResults:TriggerResultInserter            1.8275e-05    3.31097e-05   0.000141587   2.0574e-05    3.62658e-05      10     
end_path:out1:RootOutput                               9.257e-06    1.53599e-05   5.3741e-05    1.14115e-05   1.28378e-05      10     
end_path:out1:RootOutput(write)                        0.300342      0.403173      0.512309      0.395524      0.050041        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 12977.9 MB
  Peak resident set size usage (VmHWM): 2621.07 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0