21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 228444.51@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 228444.51@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7875 | |
| Stage ID | 1 | |
| User name | higuera@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-24 11:25:08 | |
| Site | UK_Durham | |
| Entry | DUNE_UK_SGridDurham_ce4 | |
| Last heartbeat | 2025-06-24 14:21:02 | |
| From worker node | Hostname | n142.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5120 CPU @ 2.20GHz | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4194304000 (4000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-24 12:03:08 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_395_20231202T190844Z_gen_g4_detsim_hitreco.root | |
| Jobscript | Exit code | 0 |
| Real time | 33m (2008s) | |
| CPU time | 20m (1222s = 60%) | |
| Max RSS bytes | 1964564480 (1873 MiB) | |
| Outputting started | 2025-06-24 12:36:37 | |
| Output files | ||
| Finished | 2025-06-24 14:21:02 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
6_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory WARNING: '/_reco/localProducts_larsoft_v09_89_01_e26_prof/' is not a directory Justin processors: 1 /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/cdba6e7604dd8b049ed4282aee20b939a4b77fad/atmo_reco2_lbl_model.fcl Input PFN = root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/6f/af/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_395_20231202T190844Z_gen_g4_detsim_hitreco.root /cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_5_5a/Linux64bit+3.10-2.17-e26-p3915-prof/lib/libXrdPosixPreload.so === Start last 100 lines of lar log file === %MSG-e ShowerProducedPtrsHolder: LArPandoraModularShowerCreation:pandoraShower@BeginModule 24-Jun-2025 13:36:05 BST run: 50571149 subRun: 1 event: 39598 Trying to add data product: InitialTrack. This element does not exist in the element holder %MSG Boundary wire vector sizes: 152, 187, 188 minwire 0: 678 minwire 1: 2013 minwire 2: 468 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.932017, 0.032167, 0.0358161, Output 1: 0.10696, 0.863416, 0.0275486, 0.00207532, Output 2: 0.0555124, 0.938982, 0.00548795, 1.79106e-05, %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 13:36:06 BST run: 50571149 subRun: 1 event: 39598 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 589 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1955! Begin processing the 99th record. run: 50571149 subRun: 1 event: 39599 at 24-Jun-2025 13:36:14 BST Boundary wire vector sizes: 87, 295, 221 minwire 0: 1828 minwire 1: 1324 minwire 2: 1581 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.99703, 0.000583884, 0.00238634, Output 1: 0.00281582, 0.988961, 0.00805511, 0.000168397, Output 2: 0.997095, 0.00283468, 6.91373e-05, 1.31449e-06, 530 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1773! Begin processing the 100th record. run: 50571149 subRun: 1 event: 39600 at 24-Jun-2025 13:36:24 BST Boundary wire vector sizes: 154, 193, 174 minwire 0: 1482 minwire 1: 1059 minwire 2: 1407 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Used alternate method to get min and max tdcs due to vertex determination failure: 0, 299 Classifier summary: Output 0: 0.00024512, 0.99884, 0.00091483, Output 1: 0.00943943, 0.986074, 0.00434866, 0.000138194, Output 2: 0.998876, 0.0010919, 3.11263e-05, 8.82252e-07, %MSG-e DUNEAna: EnergyReco:energyrecnumumcs@BeginModule 24-Jun-2025 13:36:28 BST run: 50571149 subRun: 1 event: 39600 Failed to find product with label wclsmcnfsp:gauss ... returning empty vector %MSG 504 ***Wiener Filter*******ApplyPostfilter*** Running Ophitfinder with InputDigiType = 'recob' Found hits: 1585! 24-Jun-2025 13:36:36 BST Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_395_20231202T190844Z_gen_g4_detsim_hitreco_reco2.root" 24-Jun-2025 13:36:36 BST Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/6f/af/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_395_20231202T190844Z_gen_g4_detsim_hitreco.root" ======================================================================================================================================== TimeTracker printout (sec) Min Avg Max Median RMS nEvts ======================================================================================================================================== Full event 0.771314 12.3784 368.402 7.29578 36.2617 100 ---------------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.0303982 0.0379767 0.119534 0.0315936 0.0149432 100 reco:pandora:StandardPandora 0.188292 6.78358 355.065 2.21063 35.2069 100 reco:pandoraTrack:LArPandoraTrackCreation 0.000120311 0.0135241 0.2997 0.00260724 0.0356599 100 reco:pandoraShower:LArPandoraModularShowerCreation 0.000172664 0.0174251 0.141911 0.00478399 0.0287062 100 reco:pandoracalo:Calorimetry 9.3878e-05 0.0469039 1.2857 0.00213965 0.157186 100 reco:pandorapid:Chi2ParticleID 3.6508e-05 0.000229459 0.00246889 0.000160644 0.000282473 100 reco:cvnmap:CVNMapper 2.0089e-05 0.0187462 0.0656263 0.0194639 0.0136443 100 reco:cvneva:CVNEvaluator 1.6698e-05 0.512573 4.56267 0.578217 0.498847 100 reco:energyrecnumu:EnergyReco 0.0899343 0.237118 1.14628 0.170938 0.215308 100 reco:energyrecnue:EnergyReco 0.000189556 0.0217993 0.160994 0.00218241 0.037035 100 reco:energyrecnc:EnergyReco 0.00232937 0.176225 0.929199 0.169024 0.143676 100 reco:energyrecnumurange:EnergyReco 0.000185868 0.0690975 0.782344 0.002315 0.156929 100 reco:energyrecnumumcs:EnergyReco 0.000115031 0.069467 0.791841 0.0023647 0.158166 100 reco:opdec:Deconvolution 0.0939304 0.458569 0.924641 0.442554 0.18473 100 reco:ophitspe:OpHitFinderDeco 0.268266 3.55336 8.88396 3.15422 2.00075 100 reco:opflash:OpFlashFinder 0.000127685 0.0017483 0.00467391 0.00162301 0.000952334 100 reco:rns:RandomNumberSaver 1.8537e-05 3.93529e-05 0.000323734 3.1667e-05 3.30408e-05 100 reco:anglereconue:NuAngularReco 0.000136887 0.00384321 0.117563 0.000425318 0.0131021 100 reco:anglereconumu:NuAngularReco 7.4013e-05 0.0035 0.102588 0.000277562 0.0118084 100 reco:anglereconuepfps:NuAngularReco 0.000111988 0.00373681 0.115036 0.000374274 0.0128695 100 reco:anglereconumupfps:NuAngularReco 0.000100239 0.00362982 0.10956 0.000333114 0.0124146 100 [art]:TriggerResults:TriggerResultInserter 1.0317e-05 1.47098e-05 5.1545e-05 1.3527e-05 5.33974e-06 100 end_path:out1:RootOutput 2.425e-06 4.7665e-06 1.8856e-05 3.3905e-06 2.51008e-06 100 end_path:out1:RootOutput(write) 0.00350219 0.34412 0.825674 0.338166 0.215653 100 ======================================================================================================================================== ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 3610.73 MB Peak resident set size usage (VmHWM): 1964.56 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === lar exit code 0