21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 210281.26@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 210281.26@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7172 | |
| Stage ID | 1 | |
| User name | calcuttj@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-05-28 10:26:56 | |
| Site | CERN | |
| Entry | CMSHTPC_T2_CH_CERN_ce506 | |
| Last heartbeat | 2025-05-28 10:35:00 | |
| From worker node | Hostname | b9p05p8888.cern.ch |
| cpuinfo | AMD EPYC 7543 32-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 343800 (95 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-05-28 10:27:38 | |
| Input files | monte-carlo-007172-000093 | |
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-05-28 10:35:00 | |
| Saved logs | justin-logs:210281.26-justin-prod-sched02.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
ephfs/experiments/dune/RSE/ehn1-beam-np04/58/dc/H4_v34b_1GeV_-27.7_003333_20250523T120552Z_003333.root
hadd Source file 38: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/81/79/H4_v34b_1GeV_-27.7_003489_20250523T100942Z_003489.root
hadd Source file 39: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/8e/6e/H4_v34b_1GeV_-27.7_003493_20250523T114941Z_003493.root
hadd Source file 40: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/80/81/H4_v34b_1GeV_-27.7_003590_20250523T093028Z_003590.root
hadd Source file 41: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/d5/99/H4_v34b_1GeV_-27.7_003640_20250523T141526Z_003640.root
hadd Source file 42: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/93/4d/H4_v34b_1GeV_-27.7_003809_20250523T130614Z_003809.root
hadd Source file 43: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/f9/d0/H4_v34b_1GeV_-27.7_003864_20250523T111838Z_003864.root
hadd Source file 44: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/69/fe/H4_v34b_1GeV_-27.7_004193_20250523T111007Z_004193.root
hadd Source file 45: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/38/5b/H4_v34b_1GeV_-27.7_004360_20250523T104053Z_004360.root
hadd Source file 46: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/7c/f0/H4_v34b_1GeV_-27.7_004475_20250523T110519Z_004475.root
hadd Source file 47: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/86/cd/H4_v34b_1GeV_-27.7_004571_20250523T103648Z_004571.root
hadd Source file 48: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/50/ed/H4_v34b_1GeV_-27.7_004590_20250523T113417Z_004590.root
hadd Source file 49: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/d2/40/H4_v34b_1GeV_-27.7_005016_20250523T124355Z_005016.root
hadd Source file 50: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/72/18/H4_v34b_1GeV_-27.7_005053_20250523T173853Z_005053.root
hadd Source file 51: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/bf/bf/H4_v34b_1GeV_-27.7_005070_20250523T104950Z_005070.root
hadd Source file 52: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/32/8b/H4_v34b_1GeV_-27.7_005112_20250523T110552Z_005112.root
hadd Source file 53: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/2c/c4/H4_v34b_1GeV_-27.7_005311_20250523T110815Z_005311.root
hadd Source file 54: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/f7/62/H4_v34b_1GeV_-27.7_005421_20250523T115618Z_005421.root
hadd Source file 55: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/90/ea/H4_v34b_1GeV_-27.7_005508_20250523T121851Z_005508.root
hadd Source file 56: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/d6/06/H4_v34b_1GeV_-27.7_005575_20250523T132804Z_005575.root
hadd Source file 57: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/8c/84/H4_v34b_1GeV_-27.7_005778_20250523T130512Z_005778.root
hadd Source file 58: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/2b/5a/H4_v34b_1GeV_-27.7_005981_20250523T223641Z_005981.root
hadd Source file 59: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/7e/ff/H4_v34b_1GeV_-27.7_006112_20250523T142626Z_006112.root
hadd Source file 60: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/31/b1/H4_v34b_1GeV_-27.7_006174_20250523T141900Z_006174.root
hadd Source file 61: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/b5/2f/H4_v34b_1GeV_-27.7_006376_20250523T161251Z_006376.root
hadd Source file 62: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/0d/c4/H4_v34b_1GeV_-27.7_006417_20250523T132954Z_006417.root
hadd Source file 63: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/aa/75/H4_v34b_1GeV_-27.7_006517_20250523T144705Z_006517.root
hadd Source file 64: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/39/30/H4_v34b_1GeV_-27.7_006553_20250523T144137Z_006553.root
hadd Source file 65: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/6d/f8/H4_v34b_1GeV_-27.7_006641_20250523T192145Z_006641.root
hadd Source file 66: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/23/c8/H4_v34b_1GeV_-27.7_006685_20250523T164222Z_006685.root
hadd Source file 67: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/72/92/H4_v34b_1GeV_-27.7_006702_20250523T140403Z_006702.root
hadd Source file 68: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/29/a8/H4_v34b_1GeV_-27.7_006842_20250523T153128Z_006842.root
hadd Source file 69: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/7b/b2/H4_v34b_1GeV_-27.7_007061_20250523T162421Z_007061.root
hadd Source file 70: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/b5/7f/H4_v34b_1GeV_-27.7_007153_20250523T235756Z_007153.root
hadd Source file 71: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/ab/5c/H4_v34b_1GeV_-27.7_007184_20250523T151957Z_007184.root
hadd Source file 72: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/1d/c3/H4_v34b_1GeV_-27.7_007234_20250523T152609Z_007234.root
hadd Source file 73: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/f2/26/H4_v34b_1GeV_-27.7_007239_20250523T162724Z_007239.root
hadd Source file 74: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/ff/7b/H4_v34b_1GeV_-27.7_007340_20250523T165104Z_007340.root
hadd Source file 75: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/03/39/H4_v34b_1GeV_-27.7_007900_20250523T172718Z_007900.root
hadd Source file 76: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/de/d6/H4_v34b_1GeV_-27.7_007946_20250523T165234Z_007946.root
hadd Source file 77: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/5c/9b/H4_v34b_1GeV_-27.7_008004_20250524T021914Z_008004.root
hadd Source file 78: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/a7/93/H4_v34b_1GeV_-27.7_008089_20250523T174224Z_008089.root
hadd Source file 79: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/17/11/H4_v34b_1GeV_-27.7_008108_20250523T180550Z_008108.root
hadd Source file 80: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/62/7c/H4_v34b_1GeV_-27.7_008200_20250524T005347Z_008200.root
hadd Source file 81: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/f8/b5/H4_v34b_1GeV_-27.7_008247_20250523T232512Z_008247.root
hadd Source file 82: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/37/55/H4_v34b_1GeV_-27.7_008280_20250523T165948Z_008280.root
hadd Source file 83: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/b2/0d/H4_v34b_1GeV_-27.7_008376_20250524T004718Z_008376.root
hadd Source file 84: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/ca/48/H4_v34b_1GeV_-27.7_008430_20250524T000749Z_008430.root
hadd Source file 85: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/eb/f1/H4_v34b_1GeV_-27.7_008438_20250524T000314Z_008438.root
hadd Source file 86: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/c8/29/H4_v34b_1GeV_-27.7_008554_20250524T002743Z_008554.root
hadd Source file 87: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/d7/d3/H4_v34b_1GeV_-27.7_008662_20250523T165000Z_008662.root
hadd Source file 88: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/12/49/H4_v34b_1GeV_-27.7_008663_20250523T224935Z_008663.root
hadd Source file 89: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/d6/22/H4_v34b_1GeV_-27.7_008686_20250524T001303Z_008686.root
hadd Source file 90: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/a6/7a/H4_v34b_1GeV_-27.7_008769_20250524T014436Z_008769.root
hadd Source file 91: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/a9/ac/H4_v34b_1GeV_-27.7_008770_20250524T002313Z_008770.root
hadd Source file 92: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/51/3e/H4_v34b_1GeV_-27.7_008782_20250524T011911Z_008782.root
hadd Source file 93: root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/42/a0/H4_v34b_1GeV_-27.7_008978_20250523T191839Z_008978.root
Error in <TNetXNGFile::Open>: [FATAL] Auth failed: No protocols left to try
Error in <TFileMerger::AddFile>: cannot open file root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/42/a0/H4_v34b_1GeV_-27.7_008978_20250523T191839Z_008978.root
hadd exiting due to error in root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/42/a0/H4_v34b_1GeV_-27.7_008978_20250523T191839Z_008978.root
Traceback (most recent call last):
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f/merge_g4bl.py", line 403, in <module>
do_merge(args)
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f/merge_g4bl.py", line 119, in do_merge
raise Exception('Error in hadd')
Exception: Error in hadd
Exiting with error