Jobsub ID 105927.47@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	105927.47@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4198
Stage ID	1
User name	osbiston@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-21 13:26:22
Site	CZ_FZU
Entry	Nova_CZ_FZU_osgce2
Last heartbeat	2024-11-21 21:25:56
From worker node	Hostname	mff-magic23.farm.particle.cz
	cpuinfo	Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	2097152000 (2000 MiB)
	Wall seconds limit	172800 (48 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-21 13:27:39
Input files	fardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T194835Z_gen_001145_supernova_g4stage1_g4stage2_detsim_reco.root
Jobscript	Exit code	0
	Real time	7h (28674s)
	CPU time	7h (27781s = 96%)
Outputting started	2024-11-21 21:25:34
Output files
Finished	2024-11-21 21:25:56
List job events Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-vd/9e/b4/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T194835Z_gen_001145_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
Boundary wire vector sizes: 6760, 6633, 6614
minwire 0: 44
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1530, 2029
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.083662, 
Output 1: 0.0660749, 0.0638715, 0.111608, 0.758445, 
Output 2: 0.0455996, 0.18466, 0.737638, 0.0321022, 
Output 3: 0.953724, 0.0221126, 0.00643705, 0.0177263, 
Output 4: 0.730937, 0.227124, 0.034562, 0.00737704, 
Output 5: 0.613646, 0.272747, 0.0870606, 0.0265465, 
Output 6: 0.0171208, 0.0475723, 0.0595652, 0.875742, 

Begin processing the 8th record. run: 3799 subRun: 0 event: 11448 at 21-Nov-2024 20:16:15 CET
Boundary wire vector sizes: 6558, 6537, 6498
minwire 0: 11
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.183364, 
Output 1: 0.122879, 0.125461, 0.142815, 0.608844, 
Output 2: 0.0405915, 0.153198, 0.775316, 0.0308944, 
Output 3: 0.987688, 0.00615827, 0.000940165, 0.00521321, 
Output 4: 0.659178, 0.311377, 0.0257567, 0.00368794, 
Output 5: 0.204933, 0.323776, 0.322695, 0.148596, 
Output 6: 0.042647, 0.0926867, 0.0949701, 0.769696, 

Begin processing the 9th record. run: 3799 subRun: 0 event: 11449 at 21-Nov-2024 20:59:14 CET
Boundary wire vector sizes: 6768, 6729, 6734
minwire 0: 14
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max wires due to vertex determination failure: 1030, 1529
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.117499, 
Output 1: 0.175245, 0.0750392, 0.258965, 0.490751, 
Output 2: 0.0419432, 0.0578719, 0.888931, 0.0112533, 
Output 3: 0.959372, 0.0368896, 0.00172295, 0.00201558, 
Output 4: 0.542961, 0.397748, 0.0467967, 0.0124944, 
Output 5: 0.161112, 0.359182, 0.314189, 0.165517, 
Output 6: 0.167024, 0.281053, 0.137481, 0.414442, 

Begin processing the 10th record. run: 3799 subRun: 0 event: 11450 at 21-Nov-2024 21:42:33 CET
Boundary wire vector sizes: 6780, 6875, 6800
minwire 0: 60
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.115783, 
Output 1: 0.0432088, 0.120894, 0.171089, 0.664808, 
Output 2: 0.041723, 0.190253, 0.74326, 0.0247637, 
Output 3: 0.946793, 0.0279656, 0.00772186, 0.0175195, 
Output 4: 0.372228, 0.544174, 0.0756507, 0.00794722, 
Output 5: 0.392576, 0.469339, 0.117606, 0.0204785, 
Output 6: 0.0260625, 0.0594665, 0.0709602, 0.843511, 

21-Nov-2024 22:25:33 CET  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T194835Z_gen_001145_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-21T_132746Z.root"
21-Nov-2024 22:25:33 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-vd/9e/b4/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241023T194835Z_gen_001145_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2090.63       2835.91       3208.56       3000.52       342.924        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00087271    0.00477785     0.0194022    0.00113616    0.00697151       10     
reco:pandora:StandardPandora                            2087.31       2832.53       3205.03       2997.91       342.733        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000304784   0.00373643     0.0321876    0.000393316   0.00949793       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000321022   0.000695163   0.00323701    0.000422824   0.000848916      10     
reco:pandoracalo:Calorimetry                          0.00159768     0.0526191     0.0818719     0.0515313     0.0217087       10     
reco:pandorapid:Chi2ParticleID                        8.7462e-05    0.000179614   0.00081887    0.000106348   0.000213494      10     
reco:cvnmap:CVNMapper                                  0.0969989     0.124008      0.222213      0.112978      0.0347895       10     
reco:cvneva:CVNEvaluator                                1.36149       2.35364       4.89627       2.15518       0.88741        10     
reco:energyrecnumu:EnergyReco                          0.137609      0.148122      0.166485      0.148521     0.00877076       10     
reco:energyrecnue:EnergyReco                           0.010613      0.0139974     0.0165032     0.0144208    0.00171655       10     
reco:energyrecnc:EnergyReco                            0.0105888     0.0138427     0.0164569     0.0144082    0.00180666       10     
[art]:TriggerResults:TriggerResultInserter            1.7225e-05    3.44484e-05   7.9166e-05    2.9817e-05    1.59678e-05      10     
end_path:out1:RootOutput                               8.692e-06    1.29986e-05   2.3885e-05    1.1053e-05    4.89938e-06      10     
end_path:out1:RootOutput(write)                        0.459368      0.662824      0.961142      0.676314      0.133384        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9752.26 MB
  Peak resident set size usage (VmHWM): 2795.1 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0