Jobsub ID 108541.1@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108541.1@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4199
Stage ID	1
User name	osbiston@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-22 22:55:11
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce509
Last heartbeat	2024-11-23 07:17:22
From worker node	Hostname	b9p08p1455.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:31:02
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241025T193614Z_gen_000177_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	2h (9961s)
	CPU time	2h (9462s = 94%)
Outputting started	2024-11-23 07:17:04
Output files
Finished	2024-11-23 07:17:22
List job events Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/fardet-vd/52/7f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241025T193614Z_gen_000177_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
minwire 0: 34
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0755248, 
Output 1: 0.0895871, 0.0643418, 0.111716, 0.734355, 
Output 2: 0.0557556, 0.205438, 0.701551, 0.0372556, 
Output 3: 0.947738, 0.0202371, 0.00712142, 0.0249035, 
Output 4: 0.824484, 0.144621, 0.0241417, 0.00675353, 
Output 5: 0.757971, 0.189782, 0.0397938, 0.0124529, 
Output 6: 0.0178117, 0.0343078, 0.0444994, 0.903381, 

Begin processing the 8th record. run: 3844 subRun: 0 event: 1768 at 23-Nov-2024 07:25:41 CET
Boundary wire vector sizes: 6733, 6688, 6683
minwire 0: 10
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 2038, 2537
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.13089, 
Output 1: 0.0918824, 0.0931966, 0.09136, 0.723561, 
Output 2: 0.0392535, 0.248995, 0.662565, 0.0491868, 
Output 3: 0.974452, 0.0106758, 0.00341677, 0.0114557, 
Output 4: 0.893593, 0.0907207, 0.0128546, 0.00283128, 
Output 5: 0.430941, 0.293065, 0.209474, 0.0665201, 
Output 6: 0.0166222, 0.0417651, 0.0580103, 0.883602, 

Begin processing the 9th record. run: 3844 subRun: 0 event: 1769 at 23-Nov-2024 07:43:30 CET
Boundary wire vector sizes: 6524, 6447, 6517
minwire 0: 38
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1381, 1880
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0906011, 
Output 1: 0.0988485, 0.144816, 0.108872, 0.647464, 
Output 2: 0.0596716, 0.27301, 0.622483, 0.0448356, 
Output 3: 0.95832, 0.0136736, 0.00475384, 0.0232523, 
Output 4: 0.889672, 0.100609, 0.00809867, 0.00162098, 
Output 5: 0.582229, 0.333887, 0.0677954, 0.0160886, 
Output 6: 0.0130145, 0.0342613, 0.0478279, 0.904896, 

Begin processing the 10th record. run: 3844 subRun: 0 event: 1770 at 23-Nov-2024 07:59:51 CET
Boundary wire vector sizes: 6795, 6721, 6697
minwire 0: 32
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 157, 656
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.101003, 
Output 1: 0.186045, 0.129809, 0.262957, 0.42119, 
Output 2: 0.0372761, 0.0608634, 0.89298, 0.00888037, 
Output 3: 0.971237, 0.0240634, 0.00142907, 0.00327061, 
Output 4: 0.606946, 0.356275, 0.0293536, 0.00742566, 
Output 5: 0.200513, 0.466804, 0.233047, 0.0996358, 
Output 6: 0.135545, 0.220512, 0.120372, 0.52357, 

23-Nov-2024 08:17:03 CET  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241025T193614Z_gen_000177_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-23T_043108Z.root"
23-Nov-2024 08:17:03 CET  Closed input file "root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/fardet-vd/52/7f/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241025T193614Z_gen_000177_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              846.187       957.945       1066.06       952.642       65.3382        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000574757    0.0033619     0.0167132    0.000816369   0.00537944       10     
reco:pandora:StandardPandora                            844.584       956.159       1064.35       951.012       65.4372        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000189082   0.00220584     0.0192712    0.000233433   0.00569097       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000172802   0.000503035   0.00309406    0.000228477   0.000863965      10     
reco:pandoracalo:Calorimetry                           0.0013182     0.016872      0.0238806     0.0182394    0.00599185       10     
reco:pandorapid:Chi2ParticleID                         4.652e-05    0.000121925   0.000656297   5.95855e-05   0.000178658      10     
reco:cvnmap:CVNMapper                                  0.0338195     0.0409164     0.0508238     0.0409392     0.0056331       10     
reco:cvneva:CVNEvaluator                                1.04488       1.25343       2.23007       1.15653      0.328118        10     
reco:energyrecnumu:EnergyReco                          0.0745253     0.0847281     0.0929822     0.0863059    0.00551235       10     
reco:energyrecnue:EnergyReco                          0.00752955    0.00860309    0.00898697    0.00881353    0.000433558      10     
reco:energyrecnc:EnergyReco                           0.00696363    0.00837566    0.00914776    0.00860546    0.000651715      10     
[art]:TriggerResults:TriggerResultInserter             1.182e-05    2.32774e-05   0.000108281    1.344e-05    2.83758e-05      10     
end_path:out1:RootOutput                               6.98e-06     1.1173e-05     3.743e-05     8.545e-06    8.78043e-06      10     
end_path:out1:RootOutput(write)                        0.235925      0.366116      0.448619      0.389346      0.0657521       10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21551.1 MB
  Peak resident set size usage (VmHWM): 2749.48 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0