21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 420813.77@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 420813.77@justin-prod-sched01.dune.hep.ac.uk | |
| Workflow ID | 7875 | |
| Stage ID | 1 | |
| User name | higuera@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-24 11:47:27 | |
| Site | UK_Durham | |
| Entry | DUNE_UK_SGridDurham_ce4 | |
| Last heartbeat | 2025-06-24 14:20:28 | |
| From worker node | Hostname | n152.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-24 11:58:56 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_16_20231202T180343Z_gen_g4_detsim_hitreco.root | |
| Jobscript | Exit code | 0 |
| Real time | 36m (2175s) | |
| CPU time | 24m (1475s = 67%) | |
| Max RSS bytes | 1903968256 (1815 MiB) | |
| Outputting started | 2025-06-24 12:35:12 | |
| Output files | ||
| Finished | 2025-06-24 14:20:28 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
NING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory Justin processors: 1 /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/cdba6e7604dd8b049ed4282aee20b939a4b77fad/atmo_reco2_lbl_model.fcl Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/02/09/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_16_20231202T180343Z_gen_g4_detsim_hitreco.root /cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so === Start last 100 lines of lar log file === Output 1: 0.897675, 0.101411, 0.000789516, 0.00012511, Output 2: 0.998992, 0.000978481, 2.6626e-05, 3.34421e-06, %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 13:32:52 BST run: 74506302 subRun: 1 event: 1698 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 257 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 677! Begin processing the 99th record. run: 74506302 subRun: 1 event: 1699 at 24-Jun-2025 13:32:56 BST %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 13:34:42 BST run: 74506302 subRun: 1 event: 1699 Trying to add data product: InitialTrack. This element does not exist in the element holder %MSG %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 13:34:42 BST run: 74506302 subRun: 1 event: 1699 Trying to add data product: ShowerPCA. This element does not exist in the element holder %MSG Boundary wire vector sizes: 6824, 6577, 4635 minwire 0: 1359 minwire 1: 110 minwire 2: 1295 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.1702, 0.591629, 0.23817, Output 1: 0.338625, 0.378475, 0.161689, 0.121212, Output 2: 0.0174727, 0.0748492, 0.235584, 0.672094, 770 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 3092! Begin processing the 100th record. run: 74506302 subRun: 1 event: 1700 at 24-Jun-2025 13:34:59 BST %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 13:35:02 BST run: 74506302 subRun: 1 event: 1700 Trying to add data product: InitialTrack. This element does not exist in the element holder %MSG %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 13:35:02 BST run: 74506302 subRun: 1 event: 1700 Trying to add data product: InitialTrack. This element does not exist in the element holder %MSG %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 13:35:02 BST run: 74506302 subRun: 1 event: 1700 Trying to add data product: ShowerPCA. This element does not exist in the element holder %MSG Boundary wire vector sizes: 198, 241, 233 minwire 0: 1164 minwire 1: 2251 minwire 2: 647 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.988803, 0.00125687, 0.0099396, Output 1: 0.00533628, 0.017107, 0.125368, 0.852189, Output 2: 0.984697, 0.0146928, 0.000531254, 7.86689e-05, 592 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 2150! 24-Jun-2025 13:35:11 BST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_16_20231202T180343Z_gen_g4_detsim_hitreco_reco2.root" 24-Jun-2025 13:35:11 BST Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/02/09/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74506302_16_20231202T180343Z_gen_g4_detsim_hitreco.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 4.29886 14.7752 209.761 9.99177 24.6689 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.0308038 0.041353 0.0954892 0.0329137 0.0153241 100 reco:pandora:StandardPandora 2.12103 6.95668 192.566 2.54692 22.6493 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000208787 0.0143614 0.224581 0.00280985 0.0354286 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000289316 0.0334532 0.725843 0.00565774 0.0955518 100 reco:pandoracalo:Calorimetry 0.000143245 0.0468933 1.17572 0.00193836 0.16526 100 reco:pandorapid:Chi2ParticleID 5.2865e-05 0.000272363 0.00183032 0.000184057 0.000287583 100 reco:cvnmap:CVNMapper 3.8061e-05 0.0206029 0.0737052 0.0204319 0.015558 100 reco:cvneva:CVNEvaluator 2.5256e-05 0.657159 4.01068 0.797338 0.504005 100 reco:energyrecnumu:EnergyReco 0.0908004 0.277304 3.0405 0.173911 0.418628 100 reco:energyrecnue:EnergyReco 0.00030846 0.0259285 0.0937045 0.00196336 0.03798 100 reco:energyrecnc:EnergyReco 0.00402745 0.175672 1.06477 0.17429 0.158715 100 reco:energyrecnumurange:EnergyReco 0.000295602 0.108933 2.66235 0.00175222 0.36165 100 reco:energyrecnumumcs:EnergyReco 0.000183749 0.107709 2.60247 0.00180292 0.35546 100 reco:opdec:Deconvolution 0.159583 0.538746 1.13514 0.519728 0.197375 100 reco:ophitspe:OpHitFinderDeco 0.94206 5.28073 10.9879 5.29441 2.11951 100 reco:opflash:OpFlashFinder 0.000569053 0.00253441 0.00774575 0.00230157 0.00126543 100 reco:rns:RandomNumberSaver 2.102e-05 6.45187e-05 0.000440838 6.13985e-05 4.4802e-05 100 reco:anglereconue:NuAngularReco 0.000193366 0.00379139 0.0709321 0.00066633 0.010463 100 reco:anglereconumu:NuAngularReco 0.000102089 0.00343158 0.068308 0.000392458 0.0100749 100 reco:anglereconuepfps:NuAngularReco 0.000130894 0.00369554 0.0694271 0.00057232 0.0104192 100 reco:anglereconumupfps:NuAngularReco 0.000129332 0.00360295 0.068699 0.000496608 0.0103381 100 [art]:TriggerResults:TriggerResultInserter 1.1326e-05 2.11283e-05 6.4047e-05 1.98355e-05 6.6475e-06 100 end_path:out1:RootOutput 3.057e-06 7.07688e-06 2.3812e-05 6.7015e-06 2.99977e-06 100 end_path:out1:RootOutput(write) 0.0241181 0.470661 1.13532 0.522514 0.262398 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3584.05 MB Peak resident set size usage (VmHWM): 1903.97 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === lar exit code 0