21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 420722.155@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 420722.155@justin-prod-sched01.dune.hep.ac.uk | |
| Workflow ID | 7875 | |
| Stage ID | 1 | |
| User name | higuera@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-24 10:35:08 | |
| Site | UK_Manchester | |
| Entry | UBoone_T2_UK_Manchester_ce02 | |
| Last heartbeat | 2025-06-24 19:25:12 | |
| From worker node | Hostname | wn1206251.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | stalled | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-24 10:53:54 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496970_317_20231202T053342Z_gen_g4_detsim_hitreco.root | |
| Outputting started | 2025-06-24 11:08:00 | |
| Output files | ||
| Finished | 2025-06-24 20:26:26 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory Justin processors: 1 /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/cdba6e7604dd8b049ed4282aee20b939a4b77fad/atmo_reco2_lbl_model.fcl Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/b6/4e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496970_317_20231202T053342Z_gen_g4_detsim_hitreco.root /cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so === Start last 100 lines of lar log file === Boundary wire vector sizes: 622, 1581, 1268 minwire 0: 1435 minwire 1: 263 minwire 2: 1290 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.998034, 0.00029988, 0.00166609, Output 1: 0.0352921, 0.305626, 0.533916, 0.125165, Output 2: 0.993029, 0.00667615, 0.00028149, 1.36366e-05, 672 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 2587! Begin processing the 99th record. run: 74496970 subRun: 1 event: 31799 at 24-Jun-2025 12:07:45 BST Boundary wire vector sizes: 243, 232, 187 minwire 0: 2440 minwire 1: 1121 minwire 2: 2173 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0235247, 0.679507, 0.296969, Output 1: 0.351359, 0.579171, 0.061773, 0.00769676, Output 2: 0.296078, 0.679142, 0.0239716, 0.000807893, %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 12:07:48 BST run: 74496970 subRun: 1 event: 31799 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 532 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1972! Begin processing the 100th record. run: 74496970 subRun: 1 event: 31800 at 24-Jun-2025 12:07:52 BST Boundary wire vector sizes: 874, 1135, 761 minwire 0: 731 minwire 1: 2153 minwire 2: 297 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.016881, 0.175612, 0.807507, Output 1: 0.791365, 0.205703, 0.00264817, 0.000283925, Output 2: 0.313544, 0.605623, 0.074933, 0.0058998, %MSG-e DUNEAna: EnergyReco:energyrecnumurange@BeginModule 24-Jun-2025 12:07:56 BST run: 74496970 subRun: 1 event: 31800 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 12:07:56 BST run: 74496970 subRun: 1 event: 31800 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 633 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 2284! 24-Jun-2025 12:07:59 BST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496970_317_20231202T053342Z_gen_g4_detsim_hitreco_reco2.root" 24-Jun-2025 12:07:59 BST Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/b6/4e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496970_317_20231202T053342Z_gen_g4_detsim_hitreco.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.54011 5.97728 54.5356 5.00046 7.27109 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.0224771 0.0308554 0.0729019 0.0238423 0.0113432 100 reco:pandora:StandardPandora 0.139111 2.71333 46.5743 1.64745 6.40452 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000107353 0.00392369 0.068959 0.000816869 0.0095396 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000161274 0.00664288 0.0735638 0.00183387 0.0115943 100 reco:pandoracalo:Calorimetry 8.8658e-05 0.0121981 0.578814 0.000555184 0.0627654 100 reco:pandorapid:Chi2ParticleID 3.4405e-05 0.000128915 0.00103323 0.000100259 0.000120373 100 reco:cvnmap:CVNMapper 1.7974e-05 0.00711091 0.0390291 0.00927477 0.00627525 100 reco:cvneva:CVNEvaluator 1.5069e-05 0.311649 2.01062 0.377878 0.289734 100 reco:energyrecnumu:EnergyReco 0.0668555 0.132855 0.615834 0.121366 0.0862013 100 reco:energyrecnue:EnergyReco 0.000178017 0.0159523 0.0954934 0.00159639 0.0282765 100 reco:energyrecnc:EnergyReco 0.00211304 0.0921526 0.180298 0.126592 0.0645888 100 reco:energyrecnumurange:EnergyReco 0.00014784 0.0162261 0.406922 0.000614046 0.0584391 100 reco:energyrecnumumcs:EnergyReco 0.000103886 0.0163333 0.413257 0.000579921 0.0592203 100 reco:opdec:Deconvolution 0.0702287 0.264277 0.454319 0.28608 0.0911423 100 reco:ophitspe:OpHitFinderDeco 0.199577 2.11073 5.07284 2.13113 1.06192 100 reco:opflash:OpFlashFinder 0.000150063 0.00121965 0.00500173 0.0011885 0.000710903 100 reco:rns:RandomNumberSaver 1.7122e-05 2.55274e-05 0.000331827 2.0785e-05 3.14312e-05 100 reco:anglereconue:NuAngularReco 0.000147258 0.000913258 0.0223683 0.000251481 0.00257111 100 reco:anglereconumu:NuAngularReco 6.6395e-05 0.000811419 0.0221096 0.000160122 0.0025532 100 reco:anglereconuepfps:NuAngularReco 0.00010578 0.000925344 0.0219861 0.000256009 0.00256387 100 reco:anglereconumupfps:NuAngularReco 9.1273e-05 0.000879039 0.0224512 0.000220226 0.00259594 100 [art]:TriggerResults:TriggerResultInserter 9.077e-06 1.23666e-05 4.8712e-05 1.0755e-05 5.23275e-06 100 end_path:out1:RootOutput 2.986e-06 3.82607e-06 1.7243e-05 3.652e-06 1.40386e-06 100 end_path:out1:RootOutput(write) 0.00462058 0.237104 0.648746 0.28938 0.15995 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3584.86 MB Peak resident set size usage (VmHWM): 1826.59 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === lar exit code 0