Jobsub ID 93033.9@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	93033.9@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	3985
Stage ID	1
User name	osbiston@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-08 13:37:54
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2024-11-08 13:40:52
From worker node	Hostname	dunegli-3992924-0-fnpc22022.fnal.gov
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	172800 (48 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-08 13:39:19
Input files	fardet-vd:prodmarley_nue_cc_flat_dunevd10kt_1x8x14_3view_30deg_20241023T131053Z_gen_000351_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	1m (78s)
	CPU time	1m (73s = 93%)
Outputting started	2024-11-08 13:40:39
Output files
Finished	2024-11-08 13:40:52
List job events Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/3c/e7/prodmarley_nue_cc_flat_dunevd10kt_1x8x14_3view_30deg_20241023T131053Z_gen_000351_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0465106, 
Output 1: 0.118613, 0.159335, 0.0402132, 0.681838, 
Output 2: 0.494, 0.265207, 0.175903, 0.0648896, 
Output 3: 0.93976, 0.0579321, 0.0012801, 0.00102808, 
Output 4: 0.943199, 0.055034, 0.00153335, 0.000234003, 
Output 5: 0.302854, 0.648479, 0.0417646, 0.00690259, 
Output 6: 0.572611, 0.35206, 0.0523441, 0.0229847, 

Begin processing the 79th record. run: 3785 subRun: 0 event: 35079 at 08-Nov-2024 13:40:34 UTC
Begin processing the 80th record. run: 3785 subRun: 0 event: 35080 at 08-Nov-2024 13:40:34 UTC
Begin processing the 81st record. run: 3785 subRun: 0 event: 35081 at 08-Nov-2024 13:40:34 UTC
Begin processing the 82nd record. run: 3785 subRun: 0 event: 35082 at 08-Nov-2024 13:40:34 UTC
Begin processing the 83rd record. run: 3785 subRun: 0 event: 35083 at 08-Nov-2024 13:40:34 UTC
Boundary wire vector sizes: 34, 29, 42
minwire 0: 286
minwire 1: 1140
minwire 2: 1150
Used alternate method to get min and max wires due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0186207, 
Output 1: 0.0881392, 0.0896379, 0.0164382, 0.805785, 
Output 2: 0.445987, 0.342033, 0.147265, 0.0647156, 
Output 3: 0.832375, 0.160127, 0.00475161, 0.00274684, 
Output 4: 0.922477, 0.075503, 0.0018486, 0.000171894, 
Output 5: 0.281364, 0.671817, 0.0417598, 0.00505908, 
Output 6: 0.732486, 0.240533, 0.0224563, 0.00452447, 

Begin processing the 84th record. run: 3785 subRun: 0 event: 35084 at 08-Nov-2024 13:40:35 UTC
Boundary wire vector sizes: 37, 36, 28
minwire 0: 285
minwire 1: 1140
minwire 2: 453
Used alternate method to get min and max wires due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0149445, 
Output 1: 0.0465192, 0.0223638, 0.0103012, 0.920816, 
Output 2: 0.600736, 0.247485, 0.101446, 0.0503324, 
Output 3: 0.689509, 0.305456, 0.00383469, 0.00119986, 
Output 4: 0.915179, 0.0817788, 0.00270477, 0.000337736, 
Output 5: 0.460018, 0.51878, 0.018794, 0.00240788, 
Output 6: 0.583266, 0.354337, 0.0474027, 0.0149946, 

Begin processing the 85th record. run: 3785 subRun: 0 event: 35085 at 08-Nov-2024 13:40:37 UTC
Begin processing the 86th record. run: 3785 subRun: 0 event: 35086 at 08-Nov-2024 13:40:37 UTC
Begin processing the 87th record. run: 3785 subRun: 0 event: 35087 at 08-Nov-2024 13:40:37 UTC
Begin processing the 88th record. run: 3785 subRun: 0 event: 35088 at 08-Nov-2024 13:40:37 UTC
Begin processing the 89th record. run: 3785 subRun: 0 event: 35089 at 08-Nov-2024 13:40:37 UTC
Begin processing the 90th record. run: 3785 subRun: 0 event: 35090 at 08-Nov-2024 13:40:37 UTC
Begin processing the 91st record. run: 3785 subRun: 0 event: 35091 at 08-Nov-2024 13:40:37 UTC
Begin processing the 92nd record. run: 3785 subRun: 0 event: 35092 at 08-Nov-2024 13:40:37 UTC
Begin processing the 93rd record. run: 3785 subRun: 0 event: 35093 at 08-Nov-2024 13:40:37 UTC
Begin processing the 94th record. run: 3785 subRun: 0 event: 35094 at 08-Nov-2024 13:40:37 UTC
Begin processing the 95th record. run: 3785 subRun: 0 event: 35095 at 08-Nov-2024 13:40:37 UTC
Begin processing the 96th record. run: 3785 subRun: 0 event: 35096 at 08-Nov-2024 13:40:37 UTC
Begin processing the 97th record. run: 3785 subRun: 0 event: 35097 at 08-Nov-2024 13:40:37 UTC
Begin processing the 98th record. run: 3785 subRun: 0 event: 35098 at 08-Nov-2024 13:40:37 UTC
Begin processing the 99th record. run: 3785 subRun: 0 event: 35099 at 08-Nov-2024 13:40:37 UTC
Begin processing the 100th record. run: 3785 subRun: 0 event: 35100 at 08-Nov-2024 13:40:37 UTC
08-Nov-2024 13:40:38 UTC  Closed output file "prodmarley_nue_cc_flat_dunevd10kt_1x8x14_3view_30deg_20241023T131053Z_gen_000351_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-08T_133925Z.root"
08-Nov-2024 13:40:38 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-vd/3c/e7/prodmarley_nue_cc_flat_dunevd10kt_1x8x14_3view_30deg_20241023T131053Z_gen_000351_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.0301888     0.265009       2.7342       0.0403125      0.49199        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000163679   0.000339905    0.0016897    0.000242694   0.000273132      100    
reco:pandora:StandardPandora                           0.0116095     0.0168175     0.0515141     0.0156013     0.0054915       100    
reco:pandoraTrack:LArPandoraTrackCreation             8.3138e-05    0.000131485    0.0023141     0.0001003    0.000220483      100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000124686   0.000193087   0.00305082    0.000149884   0.000289284      100    
reco:pandoracalo:Calorimetry                          7.7797e-05    0.000120397    0.0015202    9.79255e-05   0.000142289      100    
reco:pandorapid:Chi2ParticleID                        2.7932e-05    4.36699e-05   0.000699283   3.41995e-05   6.62906e-05      100    
reco:cvnmap:CVNMapper                                 1.3716e-05    0.00274245     0.0475969    2.11755e-05   0.00663041       100    
reco:cvneva:CVNEvaluator                              1.1923e-05     0.171734       2.24611     1.64515e-05    0.382322        100    
reco:energyrecnumu:EnergyReco                          0.0123766     0.0176761     0.0280261     0.0169904    0.00457255       100    
reco:energyrecnue:EnergyReco                          0.00213228    0.00279741    0.00395181     0.0022823    0.000709321      100    
reco:energyrecnc:EnergyReco                           0.00203922     0.0026729    0.00385639    0.00219236    0.00068734       100    
[art]:TriggerResults:TriggerResultInserter             7.284e-06    1.13214e-05   7.7777e-05     9.844e-06    7.0526e-06       100    
end_path:out1:RootOutput                               2.024e-06    3.20616e-06   2.0599e-05     2.811e-06    2.12017e-06      100    
end_path:out1:RootOutput(write)                       0.000439131    0.0493663     0.372501     0.000629646    0.105534        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 20584 MB
  Peak resident set size usage (VmHWM): 1908.49 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0