21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 420711.49@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 420711.49@justin-prod-sched01.dune.hep.ac.uk | |
| Workflow ID | 7875 | |
| Stage ID | 1 | |
| User name | higuera@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-24 10:22:38 | |
| Site | UK_Manchester | |
| Entry | UBoone_T2_UK_Manchester_ce02 | |
| Last heartbeat | 2025-06-24 19:24:51 | |
| From worker node | Hostname | wn2204250.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7513 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | stalled | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-24 10:24:24 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_75_20231202T160523Z_gen_g4_detsim_hitreco.root | |
| Outputting started | 2025-06-24 10:43:01 | |
| Output files | ||
| Finished | 2025-06-24 20:26:26 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
G: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory Justin processors: 1 /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/cdba6e7604dd8b049ed4282aee20b939a4b77fad/atmo_reco2_lbl_model.fcl Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/4e/e4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_75_20231202T160523Z_gen_g4_detsim_hitreco.root /cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so === Start last 100 lines of lar log file === Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0460878, 0.450499, 0.503414, Output 1: 0.0794634, 0.456306, 0.417577, 0.0466534, Output 2: 0.0919159, 0.418841, 0.409766, 0.0794767, %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 11:42:47 BST run: 66004350 subRun: 1 event: 7597 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 699 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 2528! Begin processing the 98th record. run: 66004350 subRun: 1 event: 7598 at 24-Jun-2025 11:42:50 BST %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 11:42:51 BST run: 66004350 subRun: 1 event: 7598 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 329 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 883! Begin processing the 99th record. run: 66004350 subRun: 1 event: 7599 at 24-Jun-2025 11:42:52 BST Boundary wire vector sizes: 54, 57, 31 minwire 0: 1590 minwire 1: 915 minwire 2: 1778 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max wires due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.0342933, 0.00477493, 0.960932, Output 1: 0.143382, 0.837556, 0.0180358, 0.00102614, Output 2: 0.951102, 0.0474215, 0.0014278, 4.85768e-05, %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 11:42:54 BST run: 66004350 subRun: 1 event: 7599 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 483 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1617! Begin processing the 100th record. run: 66004350 subRun: 1 event: 7600 at 24-Jun-2025 11:42:56 BST Boundary wire vector sizes: 284, 341, 305 minwire 0: 226 minwire 1: 2230 minwire 2: 17 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00725988, 0.0141417, 0.978598, Output 1: 0.605491, 0.385451, 0.00478231, 0.00427592, Output 2: 0.953953, 0.0441755, 0.00160027, 0.000270746, 503 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1493! 24-Jun-2025 11:43:00 BST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_75_20231202T160523Z_gen_g4_detsim_hitreco_reco2.root" 24-Jun-2025 11:43:00 BST Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/4e/e4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66004350_75_20231202T160523Z_gen_g4_detsim_hitreco.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.589962 3.51789 11.5946 3.38543 1.66459 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.0224517 0.0337738 0.129819 0.0270122 0.017166 100 reco:pandora:StandardPandora 0.138756 1.25365 7.15434 1.05888 0.939438 100 reco:pandoraTrack:LArPandoraTrackCreation 9.3326e-05 0.0030311 0.0361232 0.000910447 0.00611018 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000130416 0.00536062 0.0279856 0.00233132 0.00707324 100 reco:pandoracalo:Calorimetry 7.8077e-05 0.00783293 0.168512 0.000692197 0.0276133 100 reco:pandorapid:Chi2ParticleID 2.8443e-05 0.000117396 0.000875547 9.1943e-05 0.000101897 100 reco:cvnmap:CVNMapper 1.7513e-05 0.00667422 0.0251177 0.00738787 0.00568913 100 reco:cvneva:CVNEvaluator 1.4357e-05 0.21023 1.45624 0.28484 0.184994 100 reco:energyrecnumu:EnergyReco 0.0667148 0.118025 0.387094 0.0985184 0.0598505 100 reco:energyrecnue:EnergyReco 0.000128422 0.0112982 0.15308 0.000702321 0.0273877 100 reco:energyrecnc:EnergyReco 0.00135648 0.0986794 0.207006 0.117869 0.0557075 100 reco:energyrecnumurange:EnergyReco 0.000123392 0.00950664 0.172555 0.000711628 0.0252744 100 reco:energyrecnumumcs:EnergyReco 8.1874e-05 0.00945345 0.172633 0.000670471 0.0252733 100 reco:opdec:Deconvolution 0.0693179 0.20247 0.346802 0.210712 0.0546795 100 reco:ophitspe:OpHitFinderDeco 0.205193 1.37395 2.66483 1.36054 0.548589 100 reco:opflash:OpFlashFinder 0.000118242 0.000950937 0.00348326 0.000913418 0.000471677 100 reco:rns:RandomNumberSaver 1.7864e-05 3.43186e-05 0.000317026 2.7537e-05 3.09747e-05 100 reco:anglereconue:NuAngularReco 0.000129664 0.000721514 0.00839423 0.00026297 0.0013149 100 reco:anglereconumu:NuAngularReco 5.0705e-05 0.0005771 0.00813657 0.000136572 0.00128421 100 reco:anglereconuepfps:NuAngularReco 8.2345e-05 0.000674059 0.00823764 0.000217865 0.00129004 100 reco:anglereconumupfps:NuAngularReco 6.8078e-05 0.000605061 0.00814008 0.000172414 0.00126752 100 [art]:TriggerResults:TriggerResultInserter 7.654e-06 1.02841e-05 3.5476e-05 9.543e-06 3.66494e-06 100 end_path:out1:RootOutput 2.275e-06 4.16254e-06 1.6531e-05 3.5265e-06 2.01484e-06 100 end_path:out1:RootOutput(write) 0.00393147 0.169458 0.34472 0.210297 0.0977627 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3650.45 MB Peak resident set size usage (VmHWM): 1983.79 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === lar exit code 0