21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 210276.123@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 210276.123@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7172 | |
| Stage ID | 1 | |
| User name | calcuttj@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-05-28 10:24:43 | |
| Site | UK_Manchester | |
| Entry | UBoone_T2_UK_Manchester_ce02 | |
| Last heartbeat | 2025-05-28 10:36:04 | |
| From worker node | Hostname | wn6219290.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-05-28 10:25:51 | |
| Input files | monte-carlo-007172-000059 | |
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-05-28 10:36:04 | |
| Saved logs | justin-logs:210276.123-justin-prod-sched02.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
8840.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/5f/91/H4_v34b_1GeV_-27.7_008939_20250524T001739Z_008939.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/c7/9b/H4_v34b_1GeV_-27.7_009045_20250524T004227Z_009045.root', 'root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/f0/40/H4_v34b_1GeV_-27.7_009304_20250523T191850Z_009304.root', 'root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/d6/a8/H4_v34b_1GeV_-27.7_009437_20250524T013231Z_009437.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/14/f2/H4_v34b_1GeV_-27.7_009442_20250524T004924Z_009442.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/71/4c/H4_v34b_1GeV_-27.7_009646_20250524T023249Z_009646.root', 'root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/90/17/H4_v34b_1GeV_-27.7_009834_20250524T024523Z_009834.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/63/e5/H4_v34b_1GeV_-27.7_009869_20250524T005151Z_009869.root', 'root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/6c/08/H4_v34b_1GeV_-27.7_009949_20250523T174751Z_009949.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/41/57/H4_v34b_1GeV_-27.7_009972_20250524T063821Z_009972.root']hadd Target file: H4_v34b_1GeV_-27.7_002238_210276_123_20250528T102558.root
hadd compression setting for all output: 1
hadd Source file 1: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/6c/68/H4_v34b_1GeV_-27.7_000011_20250523T124629Z_000011.root
hadd Source file 2: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/b3/92/H4_v34b_1GeV_-27.7_000152_20250523T121532Z_000152.root
hadd Source file 3: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/60/b6/H4_v34b_1GeV_-27.7_000181_20250523T111926Z_000181.root
hadd Source file 4: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/a1/53/H4_v34b_1GeV_-27.7_000250_20250523T103208Z_000250.root
hadd Source file 5: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/d8/b8/H4_v34b_1GeV_-27.7_000429_20250523T110350Z_000429.root
hadd Source file 6: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/f5/f0/H4_v34b_1GeV_-27.7_000447_20250523T124403Z_000447.root
hadd Source file 7: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/90/19/H4_v34b_1GeV_-27.7_000454_20250523T123647Z_000454.root
hadd Source file 8: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/65/96/H4_v34b_1GeV_-27.7_000719_20250523T124050Z_000719.root
hadd Source file 9: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/82/d6/H4_v34b_1GeV_-27.7_000777_20250523T094122Z_000777.root
hadd Source file 10: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/40/03/H4_v34b_1GeV_-27.7_001212_20250523T132047Z_001212.root
hadd Source file 11: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/2b/69/H4_v34b_1GeV_-27.7_001424_20250523T113535Z_001424.root
hadd Source file 12: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/09/fe/H4_v34b_1GeV_-27.7_001737_20250523T125002Z_001737.root
hadd Source file 13: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/a4/0e/H4_v34b_1GeV_-27.7_001856_20250523T095438Z_001856.root
hadd Source file 14: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/54/2e/H4_v34b_1GeV_-27.7_001859_20250523T115437Z_001859.root
hadd Source file 15: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/e9/9b/H4_v34b_1GeV_-27.7_001903_20250523T100615Z_001903.root
hadd Source file 16: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/8a/08/H4_v34b_1GeV_-27.7_001959_20250523T115630Z_001959.root
hadd Source file 17: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/e8/aa/H4_v34b_1GeV_-27.7_002021_20250523T094109Z_002021.root
hadd Source file 18: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/6c/7a/H4_v34b_1GeV_-27.7_002230_20250523T082939Z_002230.root
hadd Source file 19: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/5e/27/H4_v34b_1GeV_-27.7_002238_20250523T110030Z_002238.root
hadd Source file 20: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/65/56/H4_v34b_1GeV_-27.7_002245_20250523T133110Z_002245.root
hadd Source file 21: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/79/22/H4_v34b_1GeV_-27.7_002258_20250523T104452Z_002258.root
hadd Source file 22: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/4d/a1/H4_v34b_1GeV_-27.7_002273_20250523T081029Z_002273.root
hadd Source file 23: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/b4/f0/H4_v34b_1GeV_-27.7_002292_20250523T095600Z_002292.root
hadd Source file 24: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/0b/94/H4_v34b_1GeV_-27.7_002375_20250523T110558Z_002375.root
hadd Source file 25: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/18/5a/H4_v34b_1GeV_-27.7_002549_20250523T120343Z_002549.root
hadd Source file 26: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/ca/38/H4_v34b_1GeV_-27.7_002622_20250523T100543Z_002622.root
hadd Source file 27: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/72/a0/H4_v34b_1GeV_-27.7_002650_20250523T095936Z_002650.root
hadd Source file 28: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/82/8b/H4_v34b_1GeV_-27.7_002836_20250523T104624Z_002836.root
hadd Source file 29: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/00/96/H4_v34b_1GeV_-27.7_002995_20250523T135034Z_002995.root
hadd Source file 30: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/58/dc/H4_v34b_1GeV_-27.7_003078_20250523T105706Z_003078.root
hadd Source file 31: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/98/6a/H4_v34b_1GeV_-27.7_003171_20250523T115819Z_003171.root
hadd Source file 32: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/cf/6c/H4_v34b_1GeV_-27.7_003308_20250523T121814Z_003308.root
hadd Source file 33: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/8e/e7/H4_v34b_1GeV_-27.7_003506_20250523T104742Z_003506.root
hadd Source file 34: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/b7/10/H4_v34b_1GeV_-27.7_003545_20250523T122756Z_003545.root
hadd Source file 35: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/9c/eb/H4_v34b_1GeV_-27.7_003588_20250523T140618Z_003588.root
hadd Source file 36: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/df/ea/H4_v34b_1GeV_-27.7_003591_20250523T135203Z_003591.root
hadd Source file 37: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/ff/ad/H4_v34b_1GeV_-27.7_003600_20250523T123434Z_003600.root
hadd Source file 38: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/91/b0/H4_v34b_1GeV_-27.7_003728_20250523T140721Z_003728.root
hadd Source file 39: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/79/ec/H4_v34b_1GeV_-27.7_003877_20250523T114606Z_003877.root
hadd Source file 40: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/64/a5/H4_v34b_1GeV_-27.7_003918_20250523T112056Z_003918.root
hadd Source file 41: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/8d/cc/H4_v34b_1GeV_-27.7_003937_20250523T130102Z_003937.root
hadd Source file 42: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/9b/4d/H4_v34b_1GeV_-27.7_003980_20250523T111606Z_003980.root
hadd Source file 43: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/56/12/H4_v34b_1GeV_-27.7_004154_20250523T112550Z_004154.root
hadd Source file 44: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/d5/53/H4_v34b_1GeV_-27.7_004239_20250523T142708Z_004239.root
hadd Source file 45: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/43/89/H4_v34b_1GeV_-27.7_004264_20250523T180658Z_004264.root
hadd Source file 46: root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/13/b8/H4_v34b_1GeV_-27.7_004323_20250523T105309Z_004323.root
Error in <TNetXNGFile::Open>: [FATAL] Auth failed: No protocols left to try
Error in <TFileMerger::AddFile>: cannot open file root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/13/b8/H4_v34b_1GeV_-27.7_004323_20250523T105309Z_004323.root
hadd exiting due to error in root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/13/b8/H4_v34b_1GeV_-27.7_004323_20250523T105309Z_004323.root
Traceback (most recent call last):
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f/merge_g4bl.py", line 403, in <module>
do_merge(args)
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f/merge_g4bl.py", line 119, in do_merge
raise Exception('Error in hadd')
Exception: Error in hadd
Exiting with error