Jobsub ID 297432.168@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	297432.168@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4146
Stage ID	1
User name	osbiston@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-19 21:22:43
Site	UK_RAL-PPD
Entry	CMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat	2024-11-20 04:03:13
From worker node	Hostname	heplnc057.pp.rl.ac.uk
	cpuinfo	Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	257400 (71 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-19 21:23:53
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T215115Z_gen_000511_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	6h (23934s)
	CPU time	6h (23348s = 97%)
Outputting started	2024-11-20 04:02:48
Output files
Finished	2024-11-20 04:03:13
List job events Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-vd/63/e0/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T215115Z_gen_000511_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
Boundary wire vector sizes: 6993, 6895, 6717
minwire 0: 55
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1969, 2468
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.123323, 
Output 1: 0.0745483, 0.104824, 0.220258, 0.600369, 
Output 2: 0.0704694, 0.244996, 0.63798, 0.0465551, 
Output 3: 0.932012, 0.0508863, 0.0080902, 0.00901185, 
Output 4: 0.330102, 0.61122, 0.0551892, 0.00348883, 
Output 5: 0.69385, 0.23373, 0.0557992, 0.0166215, 
Output 6: 0.022231, 0.0619361, 0.0733033, 0.84253, 

Begin processing the 8th record. run: 3799 subRun: 0 event: 5108 at 20-Nov-2024 02:03:06 GMT
Boundary wire vector sizes: 6726, 6721, 6601
minwire 0: 23
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.097232, 
Output 1: 0.0482169, 0.0687914, 0.0866768, 0.796315, 
Output 2: 0.0463915, 0.291496, 0.637724, 0.0243887, 
Output 3: 0.965917, 0.02482, 0.00301013, 0.00625312, 
Output 4: 0.321339, 0.588681, 0.084487, 0.00549396, 
Output 5: 0.648987, 0.291645, 0.0505179, 0.00884927, 
Output 6: 0.0169364, 0.048301, 0.0673543, 0.867408, 

Begin processing the 9th record. run: 3799 subRun: 0 event: 5109 at 20-Nov-2024 02:39:32 GMT
Boundary wire vector sizes: 6660, 6662, 6477
minwire 0: 47
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1546, 2045
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.156452, 
Output 1: 0.0950916, 0.089978, 0.221188, 0.593742, 
Output 2: 0.0411303, 0.106073, 0.833627, 0.0191693, 
Output 3: 0.97704, 0.0168782, 0.0018495, 0.0042322, 
Output 4: 0.46973, 0.477074, 0.0465836, 0.00661206, 
Output 5: 0.191997, 0.43771, 0.281173, 0.0891202, 
Output 6: 0.0496366, 0.113403, 0.100424, 0.736536, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 5110 at 20-Nov-2024 03:21:14 GMT
Boundary wire vector sizes: 6725, 6684, 6616
minwire 0: 50
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.10658, 
Output 1: 0.0397737, 0.0400152, 0.128232, 0.79198, 
Output 2: 0.179145, 0.260079, 0.488117, 0.0726584, 
Output 3: 0.966153, 0.0185984, 0.00368903, 0.0115591, 
Output 4: 0.321112, 0.632218, 0.0434808, 0.00318886, 
Output 5: 0.833574, 0.153481, 0.0101494, 0.00279602, 
Output 6: 0.014245, 0.0444072, 0.0561321, 0.885216, 

20-Nov-2024 04:02:47 GMT  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T215115Z_gen_000511_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-19T_212358Z.root"
20-Nov-2024 04:02:47 GMT  Closed input file "root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/fardet-vd/63/e0/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T215115Z_gen_000511_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2179.68       2373.11       2578.02       2358.04       135.55         10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000722989   0.00699574     0.0314421    0.00114294     0.0112861       10     
reco:pandora:StandardPandora                            2174.2        2367.25       2572.26       2350.68       135.176        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000393592   0.00531077     0.0466392    0.000538129    0.0137853       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000288958   0.000980339   0.00593544    0.000445452   0.00165378       10     
reco:pandoracalo:Calorimetry                          0.00298884     0.0395625     0.0577437     0.0420767     0.0148543       10     
reco:pandorapid:Chi2ParticleID                        7.5134e-05    0.000258711   0.00165812    0.000107945   0.000466697      10     
reco:cvnmap:CVNMapper                                  0.0781796     0.123387      0.232168      0.114555      0.0415991       10     
reco:cvneva:CVNEvaluator                                3.54363       4.8107        8.71908       4.55979       1.44048        10     
reco:energyrecnumu:EnergyReco                          0.187131      0.208714      0.248912      0.202316      0.0211543       10     
reco:energyrecnue:EnergyReco                           0.0135756     0.0166602     0.0378121     0.0141588    0.00708967       10     
reco:energyrecnc:EnergyReco                            0.0135263     0.0150387     0.0212067     0.0140865    0.00222042       10     
[art]:TriggerResults:TriggerResultInserter            2.7279e-05    3.83823e-05   0.00011158    2.94305e-05   2.45497e-05      10     
end_path:out1:RootOutput                              1.0213e-05    1.53541e-05   3.8202e-05    1.18005e-05   8.13503e-06      10     
end_path:out1:RootOutput(write)                        0.543305      0.632604      0.760003       0.61289      0.0639637       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8692.61 MB
  Peak resident set size usage (VmHWM): 2814.07 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0