21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 210276.43@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 210276.43@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7172 | |
| Stage ID | 1 | |
| User name | calcuttj@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-05-28 10:24:43 | |
| Site | BR_CBPF | |
| Entry | DUNE_BR_CBPF_ce03 | |
| Last heartbeat | 2025-05-28 10:35:34 | |
| From worker node | Hostname | wn71 |
| cpuinfo | AMD EPYC 7713P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-05-28 10:26:32 | |
| Input files | monte-carlo-007172-000006 | |
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-05-28 10:35:34 | |
| Saved logs | justin-logs:210276.43-justin-prod-sched02.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/93/f6/H4_v34b_1GeV_-27.7_008194_20250524T005035Z_008194.root', 'root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/27/3f/H4_v34b_1GeV_-27.7_008204_20250523T163748Z_008204.root', 'root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/f0/84/H4_v34b_1GeV_-27.7_008305_20250523T170947Z_008305.root', 'root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/35/84/H4_v34b_1GeV_-27.7_008514_20250523T170531Z_008514.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/cf/9a/H4_v34b_1GeV_-27.7_008519_20250523T231142Z_008519.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/83/27/H4_v34b_1GeV_-27.7_008524_20250524T002336Z_008524.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/08/59/H4_v34b_1GeV_-27.7_008551_20250524T014256Z_008551.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/fc/36/H4_v34b_1GeV_-27.7_008565_20250524T015012Z_008565.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/e7/43/H4_v34b_1GeV_-27.7_008587_20250524T020724Z_008587.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/a2/29/H4_v34b_1GeV_-27.7_008601_20250524T021847Z_008601.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/64/0e/H4_v34b_1GeV_-27.7_008605_20250524T014126Z_008605.root', 'root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/3b/08/H4_v34b_1GeV_-27.7_008624_20250523T180838Z_008624.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/10/bc/H4_v34b_1GeV_-27.7_008647_20250523T181208Z_008647.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/f9/2f/H4_v34b_1GeV_-27.7_008656_20250524T001108Z_008656.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/63/23/H4_v34b_1GeV_-27.7_008715_20250524T001150Z_008715.root', 'root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/64/04/H4_v34b_1GeV_-27.7_008755_20250524T021254Z_008755.root', 'root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/f3/3b/H4_v34b_1GeV_-27.7_008918_20250524T020004Z_008918.root', 'root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/ehn1-beam-np04/02/66/H4_v34b_1GeV_-27.7_009043_20250524T002001Z_009043.root', 'root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/fa/91/H4_v34b_1GeV_-27.7_009125_20250524T015845Z_009125.root', 'root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/89/72/H4_v34b_1GeV_-27.7_009450_20250524T011459Z_009450.root', 'root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/78/6e/H4_v34b_1GeV_-27.7_009547_20250523T170413Z_009547.root', 'root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/c1/cb/H4_v34b_1GeV_-27.7_009594_20250524T020359Z_009594.root', 'root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/65/5f/H4_v34b_1GeV_-27.7_009731_20250524T023804Z_009731.root', 'root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/09/e8/H4_v34b_1GeV_-27.7_009737_20250524T015111Z_009737.root', 'root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/80/16/H4_v34b_1GeV_-27.7_009924_20250523T201635Z_009924.root', 'root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/bc/8c/H4_v34b_1GeV_-27.7_009987_20250523T205951Z_009987.root']hadd Target file: H4_v34b_1GeV_-27.7_007656_210276_43_20250528T102848.root
hadd compression setting for all output: 1
hadd Source file 1: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/69/10/H4_v34b_1GeV_-27.7_000012_20250523T104031Z_000012.root
hadd Source file 2: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/b3/7c/H4_v34b_1GeV_-27.7_000215_20250523T105405Z_000215.root
hadd Source file 3: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/97/c1/H4_v34b_1GeV_-27.7_000226_20250523T123528Z_000226.root
hadd Source file 4: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/24/e1/H4_v34b_1GeV_-27.7_000238_20250523T103400Z_000238.root
hadd Source file 5: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/8c/c4/H4_v34b_1GeV_-27.7_000378_20250523T125926Z_000378.root
hadd Source file 6: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/aa/9a/H4_v34b_1GeV_-27.7_000508_20250523T123051Z_000508.root
hadd Source file 7: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/28/ef/H4_v34b_1GeV_-27.7_000524_20250523T125545Z_000524.root
hadd Source file 8: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/1c/84/H4_v34b_1GeV_-27.7_000605_20250523T091812Z_000605.root
hadd Source file 9: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/6f/e5/H4_v34b_1GeV_-27.7_000624_20250523T103729Z_000624.root
hadd Source file 10: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/f0/1a/H4_v34b_1GeV_-27.7_000676_20250523T103422Z_000676.root
hadd Source file 11: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/32/55/H4_v34b_1GeV_-27.7_000829_20250523T091837Z_000829.root
hadd Source file 12: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/3f/52/H4_v34b_1GeV_-27.7_000847_20250523T091306Z_000847.root
hadd Source file 13: root://xgate.hec.lancs.ac.uk:1094//cephfs/grid/dune/ehn1-beam-np04/68/7d/H4_v34b_1GeV_-27.7_000980_20250523T102909Z_000980.root
hadd Source file 14: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/84/78/H4_v34b_1GeV_-27.7_001223_20250523T112302Z_001223.root
hadd Source file 15: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/ba/ef/H4_v34b_1GeV_-27.7_001247_20250523T132103Z_001247.root
hadd Source file 16: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/1a/6a/H4_v34b_1GeV_-27.7_001270_20250523T131446Z_001270.root
hadd Source file 17: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/ed/34/H4_v34b_1GeV_-27.7_001310_20250523T113440Z_001310.root
hadd Source file 18: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/d6/a6/H4_v34b_1GeV_-27.7_001405_20250523T131751Z_001405.root
hadd Source file 19: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/b9/44/H4_v34b_1GeV_-27.7_001456_20250523T114619Z_001456.root
hadd Source file 20: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/5d/7b/H4_v34b_1GeV_-27.7_001500_20250523T133229Z_001500.root
hadd Source file 21: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/d3/c6/H4_v34b_1GeV_-27.7_001534_20250523T132916Z_001534.root
hadd Source file 22: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/59/ac/H4_v34b_1GeV_-27.7_001908_20250523T095058Z_001908.root
hadd Source file 23: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/c3/aa/H4_v34b_1GeV_-27.7_002025_20250523T115459Z_002025.root
hadd Source file 24: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/df/2a/H4_v34b_1GeV_-27.7_002056_20250523T132542Z_002056.root
hadd Source file 25: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/c4/bc/H4_v34b_1GeV_-27.7_002249_20250523T083411Z_002249.root
hadd Source file 26: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/80/e3/H4_v34b_1GeV_-27.7_002329_20250523T114356Z_002329.root
hadd Source file 27: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/20/4f/H4_v34b_1GeV_-27.7_002392_20250523T120519Z_002392.root
hadd Source file 28: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/c0/37/H4_v34b_1GeV_-27.7_002565_20250523T112453Z_002565.root
hadd Source file 29: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/77/38/H4_v34b_1GeV_-27.7_002570_20250523T133727Z_002570.root
hadd Source file 30: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/c7/13/H4_v34b_1GeV_-27.7_002964_20250523T115839Z_002964.root
hadd Source file 31: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/08/33/H4_v34b_1GeV_-27.7_003037_20250523T120836Z_003037.root
hadd Source file 32: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/0f/c5/H4_v34b_1GeV_-27.7_003065_20250523T101827Z_003065.root
hadd Source file 33: root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/89/a2/H4_v34b_1GeV_-27.7_003110_20250523T111004Z_003110.root
Error in <TNetXNGFile::Open>: [FATAL] Auth failed: No protocols left to try
Error in <TFileMerger::AddFile>: cannot open file root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/89/a2/H4_v34b_1GeV_-27.7_003110_20250523T111004Z_003110.root
hadd exiting due to error in root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/89/a2/H4_v34b_1GeV_-27.7_003110_20250523T111004Z_003110.root
Traceback (most recent call last):
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f/merge_g4bl.py", line 403, in <module>
do_merge(args)
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f/merge_g4bl.py", line 119, in do_merge
raise Exception('Error in hadd')
Exception: Error in hadd
Exiting with error