justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 108603.3@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID108603.3@justin-prod-sched02.dune.hep.ac.uk
Workflow ID4199
Stage ID1
User nameosbiston@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2024-11-22 23:30:40
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce515
Last heartbeat2024-11-23 07:10:25
From worker nodeHostnameb9p11p1657.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes2097152000 (2000 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-23 04:30:14
Input filesfardet-vd:prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241025T193729Z_gen_000205_supernova_g4stage1_g4stage2_detsim_reco.root
JobscriptExit code0
Real time2h (9589s)
CPU time2h (9351s = 97%)
Outputting started2024-11-23 07:10:04
Output files
Finished2024-11-23 07:10:25
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/fardet-vd/93/f9/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241025T193729Z_gen_000205_supernova_g4stage1_g4stage2_detsim_reco.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===

Begin processing the 7th record. run: 3844 subRun: 0 event: 2047 at 23-Nov-2024 07:03:33 CET
Boundary wire vector sizes: 6903, 6827, 6702
minwire 0: 27
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.137459, 
Output 1: 0.089232, 0.0805445, 0.169328, 0.660895, 
Output 2: 0.0305023, 0.179569, 0.762276, 0.0276531, 
Output 3: 0.964075, 0.0144907, 0.00515576, 0.0162789, 
Output 4: 0.505554, 0.433106, 0.0552624, 0.00607756, 
Output 5: 0.394676, 0.39228, 0.17368, 0.0393643, 
Output 6: 0.0211121, 0.0455027, 0.0585237, 0.874861, 

Begin processing the 8th record. run: 3844 subRun: 0 event: 2048 at 23-Nov-2024 07:18:20 CET
Boundary wire vector sizes: 6804, 6762, 6638
minwire 0: 27
minwire 1: 0
minwire 2: 1
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.0659184, 
Output 1: 0.0416271, 0.0799599, 0.105622, 0.772791, 
Output 2: 0.128691, 0.260586, 0.582798, 0.0279243, 
Output 3: 0.950714, 0.0406002, 0.003149, 0.00553641, 
Output 4: 0.31563, 0.613282, 0.0651085, 0.00597998, 
Output 5: 0.772285, 0.215248, 0.0104362, 0.00203053, 
Output 6: 0.0322162, 0.104104, 0.0981513, 0.765529, 

Begin processing the 9th record. run: 3844 subRun: 0 event: 2049 at 23-Nov-2024 07:34:37 CET
Boundary wire vector sizes: 6673, 6646, 6686
minwire 0: 9
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.156515, 
Output 1: 0.122766, 0.312636, 0.180789, 0.383808, 
Output 2: 0.0256836, 0.223779, 0.710981, 0.0395563, 
Output 3: 0.958286, 0.0155011, 0.00732138, 0.0188912, 
Output 4: 0.814351, 0.163882, 0.0190309, 0.00273652, 
Output 5: 0.324918, 0.32197, 0.270488, 0.0826247, 
Output 6: 0.05011, 0.0756214, 0.076934, 0.797335, 

Begin processing the 10th record. run: 3844 subRun: 0 event: 2050 at 23-Nov-2024 07:50:13 CET
Boundary wire vector sizes: 6870, 6778, 6636
minwire 0: 57
minwire 1: 0
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 499
Classifier summary: 
Output 0: 0.098207, 
Output 1: 0.0314201, 0.0260086, 0.120744, 0.821828, 
Output 2: 0.133367, 0.273612, 0.54237, 0.0506516, 
Output 3: 0.941181, 0.0477685, 0.00521716, 0.00583366, 
Output 4: 0.130106, 0.794293, 0.0725581, 0.00304261, 
Output 5: 0.803057, 0.179104, 0.0147583, 0.00308078, 
Output 6: 0.0166005, 0.0601145, 0.0728486, 0.850437, 

23-Nov-2024 08:10:03 CET  Closed output file "prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241025T193729Z_gen_000205_supernova_g4stage1_g4stage2_detsim_reco_matt_2024-11-23T_043020Z.root"
23-Nov-2024 08:10:03 CET  Closed input file "root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/fardet-vd/93/f9/prodmarley_nue_cc_flat_radiological_decay0_dunevd10kt_1x8x14_3view_30deg_20241025T193729Z_gen_000205_supernova_g4stage1_g4stage2_detsim_reco.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              816.45        947.593       1185.83       953.215       95.0049        10     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000560405   0.00306419     0.0124129    0.000779271   0.00398117       10     
reco:pandora:StandardPandora                            814.945       945.79        1183.98       950.737       94.9241        10     
reco:pandoraTrack:LArPandoraTrackCreation             0.000172561   0.00281177     0.0242666    0.000249812   0.00717115       10     
reco:pandoraShower:LArPandoraModularShowerCreation    0.000183851   0.00058369    0.00341443    0.000275413   0.000948814      10     
reco:pandoracalo:Calorimetry                          0.00119191     0.0153811     0.0208175     0.0173193    0.00569125       10     
reco:pandorapid:Chi2ParticleID                        4.8121e-05    0.000120663   0.000603905   6.75055e-05   0.000161631      10     
reco:cvnmap:CVNMapper                                  0.0417459     0.0551094     0.0642654     0.059119     0.00761369       10     
reco:cvneva:CVNEvaluator                                1.01143       1.27804       2.6939        1.1266       0.474935        10     
reco:energyrecnumu:EnergyReco                          0.0640356     0.0946247     0.164831      0.0889867     0.0248321       10     
reco:energyrecnue:EnergyReco                          0.00587259     0.0086108    0.00964033    0.00908336    0.00130749       10     
reco:energyrecnc:EnergyReco                           0.00558147    0.00845906    0.00953788    0.00905952    0.00140015       10     
[art]:TriggerResults:TriggerResultInserter            1.0501e-05    2.05612e-05   7.7781e-05    1.4685e-05    1.91848e-05      10     
end_path:out1:RootOutput                               5.92e-06     1.0164e-05     3.06e-05      8.195e-06    6.89591e-06      10     
end_path:out1:RootOutput(write)                        0.225405      0.335698      0.496826      0.314453      0.088698        10     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21568.7 MB
  Peak resident set size usage (VmHWM): 2768.55 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0
justIN time: 2024-11-23 13:51:55 UTC       justIN version: 01.01.09