21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 228344.149@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 228344.149@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7875 | |
| Stage ID | 1 | |
| User name | higuera@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-24 09:52:54 | |
| Site | UK_Manchester | |
| Entry | UBoone_T2_UK_Manchester_ce02 | |
| Last heartbeat | 2025-06-24 19:24:39 | |
| From worker node | Hostname | wn2207291.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7513 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | stalled | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-24 10:51:58 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_854_20231201T213320Z_gen_g4_detsim_hitreco.root | |
| Outputting started | 2025-06-24 11:04:03 | |
| Output files | ||
| Finished | 2025-06-24 20:26:26 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
Products_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory Justin processors: 1 /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/cdba6e7604dd8b049ed4282aee20b939a4b77fad/atmo_reco2_lbl_model.fcl Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/23/74/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_854_20231201T213320Z_gen_g4_detsim_hitreco.root /cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so === Start last 100 lines of lar log file === Running Ophitfinder with InputDigiType = 'recob' Found hits: 3566! Begin processing the 98th record. run: 74496296 subRun: 1 event: 85498 at 24-Jun-2025 12:03:42 BST %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 12:03:43 BST run: 74496296 subRun: 1 event: 85498 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 536 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1773! Begin processing the 99th record. run: 74496296 subRun: 1 event: 85499 at 24-Jun-2025 12:03:48 BST %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 12:03:50 BST run: 74496296 subRun: 1 event: 85499 Trying to add data product: InitialTrack. This element does not exist in the element holder %MSG %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 12:03:50 BST run: 74496296 subRun: 1 event: 85499 Trying to add data product: ShowerPCA. This element does not exist in the element holder %MSG Boundary wire vector sizes: 730, 554, 566 minwire 0: 1228 minwire 1: 507 minwire 2: 1408 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.995358, 0.000657599, 0.0039843, Output 1: 0.446068, 0.361625, 0.140326, 0.0519807, Output 2: 0.70141, 0.274038, 0.022754, 0.00179843, 595 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1917! Begin processing the 100th record. run: 74496296 subRun: 1 event: 85500 at 24-Jun-2025 12:03:55 BST %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 12:03:57 BST run: 74496296 subRun: 1 event: 85500 Trying to add data product: InitialTrack. This element does not exist in the element holder %MSG %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 12:03:57 BST run: 74496296 subRun: 1 event: 85500 Trying to add data product: ShowerPCA. This element does not exist in the element holder %MSG Boundary wire vector sizes: 299, 232, 278 minwire 0: 665 minwire 1: 2341 minwire 2: 120 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00395661, 0.00454798, 0.991495, Output 1: 0.485993, 0.506176, 0.00714818, 0.000682378, Output 2: 0.991453, 0.00829038, 0.000240179, 1.6003e-05, %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 12:03:58 BST run: 74496296 subRun: 1 event: 85500 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 486 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1637! 24-Jun-2025 12:04:02 BST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_854_20231201T213320Z_gen_g4_detsim_hitreco_reco2.root" 24-Jun-2025 12:04:02 BST Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/23/74/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74496296_854_20231201T213320Z_gen_g4_detsim_hitreco.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.729099 4.60579 19.7475 4.05958 2.61564 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.022458 0.0276127 0.0666833 0.0235488 0.00944211 100 reco:pandora:StandardPandora 0.144745 1.56526 13.2917 1.12962 1.70516 100 reco:pandoraTrack:LArPandoraTrackCreation 9.2153e-05 0.00385905 0.0472432 0.00146526 0.00813563 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000124584 0.00983813 0.260229 0.00304877 0.0270928 100 reco:pandoracalo:Calorimetry 7.5301e-05 0.012674 0.361893 0.00124738 0.0455047 100 reco:pandorapid:Chi2ParticleID 2.7742e-05 0.000131322 0.0010578 0.000103154 0.000123371 100 reco:cvnmap:CVNMapper 1.8816e-05 0.00939524 0.0477204 0.00830454 0.00815047 100 reco:cvneva:CVNEvaluator 1.3976e-05 0.282493 1.54328 0.327796 0.20481 100 reco:energyrecnumu:EnergyReco 0.0668095 0.146899 1.75117 0.114275 0.176952 100 reco:energyrecnue:EnergyReco 0.00018679 0.0101475 0.0755203 0.000896407 0.0221008 100 reco:energyrecnc:EnergyReco 0.00154932 0.100964 0.228383 0.123737 0.0589128 100 reco:energyrecnumurange:EnergyReco 0.000159169 0.0281781 1.18803 0.00132964 0.1245 100 reco:energyrecnumumcs:EnergyReco 0.000103724 0.0270197 1.18266 0.00151134 0.123293 100 reco:opdec:Deconvolution 0.0821592 0.231964 0.429175 0.235812 0.0634493 100 reco:ophitspe:OpHitFinderDeco 0.378133 1.93642 4.19329 1.85876 0.837498 100 reco:opflash:OpFlashFinder 0.000358843 0.00122697 0.00350646 0.00110221 0.000574455 100 reco:rns:RandomNumberSaver 1.571e-05 4.08038e-05 0.000251743 3.734e-05 2.98674e-05 100 reco:anglereconue:NuAngularReco 0.000135174 0.000892792 0.0103067 0.000343654 0.00178578 100 reco:anglereconumu:NuAngularReco 6.6084e-05 0.000723705 0.0101445 0.000209333 0.00172606 100 reco:anglereconuepfps:NuAngularReco 9.7573e-05 0.000828045 0.00962162 0.000302112 0.00171356 100 reco:anglereconumupfps:NuAngularReco 8.2975e-05 0.000749787 0.00876742 0.000249042 0.00165417 100 [art]:TriggerResults:TriggerResultInserter 7.404e-06 1.13768e-05 4.6768e-05 1.0024e-05 5.88506e-06 100 end_path:out1:RootOutput 2.204e-06 4.50068e-06 2.4846e-05 4.409e-06 2.73973e-06 100 end_path:out1:RootOutput(write) 0.0128479 0.207469 0.411983 0.227947 0.10911 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3672.57 MB Peak resident set size usage (VmHWM): 2002.21 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === lar exit code 0