Jobsub ID 105927.35@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	105927.35@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4198
Stage ID	1
User name	osbiston@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-21 13:26:22
Site	UK_RAL-PPD
Entry	CMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat	2024-11-21 20:57:33
From worker node	Hostname	heplnc081.pp.rl.ac.uk
	cpuinfo	Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	257400 (71 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-21 13:27:38
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T211128Z_gen_002247_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	7h (26823s)
	CPU time	6h (23430s = 87%)
Outputting started	2024-11-21 20:54:42
Output files
Finished	2024-11-21 20:57:33
List job events Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-vd/0f/74/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T211128Z_gen_002247_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2076, 2575
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0782408, 
Output 1: 0.0344469, 0.102445, 0.128858, 0.73425, 
Output 2: 0.0566476, 0.250761, 0.659328, 0.0332639, 
Output 3: 0.919612, 0.0515974, 0.0132494, 0.0155415, 
Output 4: 0.650459, 0.300975, 0.0425584, 0.0060076, 
Output 5: 0.535862, 0.393533, 0.0618177, 0.00878724, 
Output 6: 0.0140197, 0.05145, 0.0649739, 0.869556, 

Begin processing the 8th record. run: 3799 subRun: 0 event: 22468 at 21-Nov-2024 18:45:00 GMT
Boundary wire vector sizes: 6571, 6564, 6619
minwire 0: 41
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 9, 508
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.11913, 
Output 1: 0.0609528, 0.0716064, 0.18862, 0.678821, 
Output 2: 0.0517164, 0.202787, 0.716086, 0.0294108, 
Output 3: 0.948088, 0.0364635, 0.00644129, 0.00900708, 
Output 4: 0.178071, 0.709518, 0.105373, 0.00703779, 
Output 5: 0.667728, 0.259279, 0.0578638, 0.0151284, 
Output 6: 0.0322189, 0.0832069, 0.0883181, 0.796256, 

Begin processing the 9th record. run: 3799 subRun: 0 event: 22469 at 21-Nov-2024 19:27:14 GMT
Boundary wire vector sizes: 6841, 6785, 6836
minwire 0: 72
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 787, 1286
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.052466, 
Output 1: 0.0305842, 0.026832, 0.0589295, 0.883654, 
Output 2: 0.182907, 0.328917, 0.443465, 0.0447115, 
Output 3: 0.926352, 0.0666309, 0.00369225, 0.0033245, 
Output 4: 0.373594, 0.570404, 0.052439, 0.00356308, 
Output 5: 0.864709, 0.128273, 0.00588222, 0.00113546, 
Output 6: 0.0248219, 0.0962441, 0.0944033, 0.784531, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 22470 at 21-Nov-2024 20:10:27 GMT
Boundary wire vector sizes: 6766, 6761, 6650
minwire 0: 57
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2366, 2865
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0645055, 
Output 1: 0.0578335, 0.0458582, 0.0473492, 0.848959, 
Output 2: 0.0561673, 0.210052, 0.702907, 0.0308735, 
Output 3: 0.974121, 0.0150284, 0.00221473, 0.0086362, 
Output 4: 0.943812, 0.0471551, 0.0064476, 0.00258524, 
Output 5: 0.536709, 0.318759, 0.111887, 0.0326445, 
Output 6: 0.0110475, 0.0336282, 0.0484178, 0.906906, 

21-Nov-2024 20:54:38 GMT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T211128Z_gen_002247_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-21T_132745Z.root"
21-Nov-2024 20:54:38 GMT  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-vd/0f/74/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T211128Z_gen_002247_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2501.66       2641.23       2853.25       2625.19       118.733        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00109959    0.00767809     0.0311499     0.0029518     0.0104475       10     
reco:pandora:StandardPandora                            2495.15       2634.14       2846.78       2618.5        118.796        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000385071   0.00491439     0.0422152    0.000515613    0.0124528       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000390599   0.00109357     0.0048384    0.000507684   0.00129882       10     
reco:pandoracalo:Calorimetry                          0.000314198    0.0356678     0.0593314     0.0400467     0.018402        10     
reco:pandorapid:Chi2ParticleID                        9.8174e-05    0.000240937   0.00122152    0.000126237   0.000328072      10     
reco:cvnmap:CVNMapper                                  0.0924889     0.164768      0.342271      0.127907      0.080734        10     
reco:cvneva:CVNEvaluator                                4.90672       5.80793       8.63161       5.37899       1.10488        10     
reco:energyrecnumu:EnergyReco                          0.229521      0.273449      0.338625      0.261934      0.0391484       10     
reco:energyrecnue:EnergyReco                           0.0134839     0.0153055     0.0264301     0.0139537    0.00374262       10     
reco:energyrecnc:EnergyReco                            0.0132818     0.0156182     0.0291715     0.0137754    0.00458321       10     
[art]:TriggerResults:TriggerResultInserter             2.894e-05    4.4905e-05     9.368e-05    4.10445e-05   1.7971e-05       10     
end_path:out1:RootOutput                              1.1164e-05    1.67447e-05    3.17e-05     1.2823e-05    6.30207e-06      10     
end_path:out1:RootOutput(write)                        0.623025       0.75816      0.973554      0.712592      0.119222        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8677.09 MB
  Peak resident set size usage (VmHWM): 2793.48 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0