21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 210276.85@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 210276.85@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7172 | |
| Stage ID | 1 | |
| User name | calcuttj@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-05-28 10:24:43 | |
| Site | UK_Manchester | |
| Entry | UBoone_T2_UK_Manchester_ce02 | |
| Last heartbeat | 2025-05-28 10:40:41 | |
| From worker node | Hostname | wn6219290.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-05-28 10:25:53 | |
| Input files | monte-carlo-007172-000061 | |
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-05-28 10:40:41 | |
| Saved logs | justin-logs:210276.85-justin-prod-sched02.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
er.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/f6/5a/H4_v34b_1GeV_-27.7_002657_20250523T133301Z_002657.root
hadd Source file 34: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/c2/58/H4_v34b_1GeV_-27.7_002711_20250523T134701Z_002711.root
hadd Source file 35: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/71/44/H4_v34b_1GeV_-27.7_002847_20250523T104321Z_002847.root
hadd Source file 36: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/55/af/H4_v34b_1GeV_-27.7_002954_20250523T103805Z_002954.root
hadd Source file 37: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/94/b4/H4_v34b_1GeV_-27.7_003005_20250523T122025Z_003005.root
hadd Source file 38: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/15/2d/H4_v34b_1GeV_-27.7_003391_20250523T115933Z_003391.root
hadd Source file 39: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/b9/57/H4_v34b_1GeV_-27.7_003412_20250523T110911Z_003412.root
hadd Source file 40: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/1c/51/H4_v34b_1GeV_-27.7_003437_20250523T083326Z_003437.root
hadd Source file 41: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/90/52/H4_v34b_1GeV_-27.7_003566_20250523T121147Z_003566.root
hadd Source file 42: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/dc/2c/H4_v34b_1GeV_-27.7_003638_20250523T140925Z_003638.root
hadd Source file 43: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/97/7f/H4_v34b_1GeV_-27.7_003857_20250523T125845Z_003857.root
hadd Source file 44: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/e6/8d/H4_v34b_1GeV_-27.7_003947_20250523T124614Z_003947.root
hadd Source file 45: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/c7/fb/H4_v34b_1GeV_-27.7_003954_20250523T113844Z_003954.root
hadd Source file 46: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/f2/03/H4_v34b_1GeV_-27.7_003974_20250523T125925Z_003974.root
hadd Source file 47: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/3e/fc/H4_v34b_1GeV_-27.7_004288_20250523T121607Z_004288.root
hadd Source file 48: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/19/74/H4_v34b_1GeV_-27.7_004391_20250523T104346Z_004391.root
hadd Source file 49: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/85/7c/H4_v34b_1GeV_-27.7_004410_20250523T141938Z_004410.root
hadd Source file 50: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/30/dc/H4_v34b_1GeV_-27.7_004534_20250523T105619Z_004534.root
hadd Source file 51: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/85/2e/H4_v34b_1GeV_-27.7_004537_20250523T113807Z_004537.root
hadd Source file 52: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/b3/18/H4_v34b_1GeV_-27.7_004542_20250523T113939Z_004542.root
hadd Source file 53: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/24/2f/H4_v34b_1GeV_-27.7_004588_20250523T104419Z_004588.root
hadd Source file 54: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/66/bb/H4_v34b_1GeV_-27.7_004660_20250523T113712Z_004660.root
hadd Source file 55: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/b5/b6/H4_v34b_1GeV_-27.7_004690_20250523T122732Z_004690.root
hadd Source file 56: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/ff/8e/H4_v34b_1GeV_-27.7_004876_20250523T115039Z_004876.root
hadd Source file 57: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/fb/d4/H4_v34b_1GeV_-27.7_004877_20250523T093114Z_004877.root
hadd Source file 58: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/2d/08/H4_v34b_1GeV_-27.7_004944_20250523T113536Z_004944.root
hadd Source file 59: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/02/83/H4_v34b_1GeV_-27.7_005007_20250523T111101Z_005007.root
hadd Source file 60: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/7d/58/H4_v34b_1GeV_-27.7_005041_20250523T122226Z_005041.root
hadd Source file 61: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/1b/22/H4_v34b_1GeV_-27.7_005217_20250523T121858Z_005217.root
hadd Source file 62: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/78/e3/H4_v34b_1GeV_-27.7_005286_20250523T143717Z_005286.root
hadd Source file 63: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/7a/ec/H4_v34b_1GeV_-27.7_005315_20250523T144716Z_005315.root
hadd Source file 64: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/c7/9d/H4_v34b_1GeV_-27.7_005365_20250523T143030Z_005365.root
hadd Source file 65: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/fe/a9/H4_v34b_1GeV_-27.7_005499_20250523T131119Z_005499.root
hadd Source file 66: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/65/41/H4_v34b_1GeV_-27.7_005523_20250523T111537Z_005523.root
hadd Source file 67: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/02/4f/H4_v34b_1GeV_-27.7_005639_20250523T104356Z_005639.root
hadd Source file 68: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/48/96/H4_v34b_1GeV_-27.7_005887_20250523T132440Z_005887.root
hadd Source file 69: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/e8/99/H4_v34b_1GeV_-27.7_005888_20250523T151716Z_005888.root
hadd Source file 70: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/6d/a3/H4_v34b_1GeV_-27.7_006021_20250523T131729Z_006021.root
hadd Source file 71: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/b1/6e/H4_v34b_1GeV_-27.7_006265_20250523T141338Z_006265.root
hadd Source file 72: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/ce/00/H4_v34b_1GeV_-27.7_006314_20250523T133007Z_006314.root
hadd Source file 73: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/db/99/H4_v34b_1GeV_-27.7_006438_20250523T143026Z_006438.root
hadd Source file 74: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/97/94/H4_v34b_1GeV_-27.7_006457_20250523T133832Z_006457.root
hadd Source file 75: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/d1/c7/H4_v34b_1GeV_-27.7_006927_20250523T142813Z_006927.root
hadd Source file 76: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/01/46/H4_v34b_1GeV_-27.7_007045_20250523T135346Z_007045.root
hadd Source file 77: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/33/0d/H4_v34b_1GeV_-27.7_007063_20250524T020622Z_007063.root
hadd Source file 78: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/dd/84/H4_v34b_1GeV_-27.7_007147_20250523T234040Z_007147.root
hadd Source file 79: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/9f/97/H4_v34b_1GeV_-27.7_007246_20250523T220837Z_007246.root
hadd Source file 80: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/91/57/H4_v34b_1GeV_-27.7_007257_20250523T155605Z_007257.root
hadd Source file 81: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/a1/fa/H4_v34b_1GeV_-27.7_007405_20250523T152610Z_007405.root
hadd Source file 82: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/53/6b/H4_v34b_1GeV_-27.7_007658_20250523T151729Z_007658.root
hadd Source file 83: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/3f/f9/H4_v34b_1GeV_-27.7_008143_20250524T001334Z_008143.root
hadd Source file 84: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/71/bb/H4_v34b_1GeV_-27.7_008167_20250524T003932Z_008167.root
hadd Source file 85: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/1b/04/H4_v34b_1GeV_-27.7_008255_20250523T165930Z_008255.root
hadd Source file 86: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/90/ae/H4_v34b_1GeV_-27.7_008429_20250524T015430Z_008429.root
hadd Source file 87: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/b0/e6/H4_v34b_1GeV_-27.7_008531_20250524T001412Z_008531.root
hadd Source file 88: root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/fe/fb/H4_v34b_1GeV_-27.7_008631_20250523T181912Z_008631.root
Error in <TNetXNGFile::Open>: [FATAL] Auth failed: No protocols left to try
Error in <TFileMerger::AddFile>: cannot open file root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/fe/fb/H4_v34b_1GeV_-27.7_008631_20250523T181912Z_008631.root
hadd exiting due to error in root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/ehn1-beam-np04/fe/fb/H4_v34b_1GeV_-27.7_008631_20250523T181912Z_008631.root
Traceback (most recent call last):
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f/merge_g4bl.py", line 403, in <module>
do_merge(args)
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f/merge_g4bl.py", line 119, in do_merge
raise Exception('Error in hadd')
Exception: Error in hadd
Exiting with error