21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 226823.8@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 226823.8@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7821 | |
| Stage ID | 1 | |
| User name | calcuttj@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-20 20:19:52 | |
| Site | UK_Manchester | |
| Entry | UBoone_T2_UK_Manchester_ce02 | |
| Last heartbeat | 2025-06-21 00:24:03 | |
| From worker node | Hostname | wn1206250.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7702P 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 4193255424 (3999 MiB) | |
| Wall seconds limit | 257400 (71 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | jobscript_error | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-20 20:26:18 | |
| Input files | monte-carlo-007821-000010 | |
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | ||
| Output files | ||
| Finished | 2025-06-21 00:24:03 | |
| Saved logs | justin-logs:226823.8-justin-prod-sched02.dune.hep.ac.uk.logs.tgz | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
np04/bd/45/H4_v34b_-1GeV_-27.7_003496_20250611T181936Z_003496.root
hadd Source file 35: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/f6/cf/H4_v34b_-1GeV_-27.7_003524_20250611T202320Z_003524.root
hadd Source file 36: root://cephc02.gla.scotgrid.ac.uk:1094//cephfs/dune/RSE/ehn1-beam-np04/c7/38/H4_v34b_-1GeV_-27.7_003674_20250611T160819Z_003674.root
hadd Source file 37: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/b3/7d/H4_v34b_-1GeV_-27.7_003702_20250612T055658Z_003702.root
hadd Source file 38: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/df/b1/H4_v34b_-1GeV_-27.7_003712_20250611T195805Z_003712.root
hadd Source file 39: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/7f/e0/H4_v34b_-1GeV_-27.7_003791_20250611T200555Z_003791.root
hadd Source file 40: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/e5/94/H4_v34b_-1GeV_-27.7_003860_20250612T050700Z_003860.root
hadd Source file 41: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/a8/5a/H4_v34b_-1GeV_-27.7_003896_20250611T222331Z_003896.root
hadd Source file 42: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/c6/05/H4_v34b_-1GeV_-27.7_004039_20250612T035607Z_004039.root
hadd Source file 43: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/6a/26/H4_v34b_-1GeV_-27.7_004043_20250611T205145Z_004043.root
hadd Source file 44: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/87/a1/H4_v34b_-1GeV_-27.7_004110_20250611T225230Z_004110.root
hadd Source file 45: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/5a/70/H4_v34b_-1GeV_-27.7_004120_20250612T043557Z_004120.root
hadd Source file 46: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/f3/bb/H4_v34b_-1GeV_-27.7_004127_20250611T203504Z_004127.root
hadd Source file 47: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/45/12/H4_v34b_-1GeV_-27.7_004220_20250611T222502Z_004220.root
hadd Source file 48: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/cc/c8/H4_v34b_-1GeV_-27.7_004288_20250612T032445Z_004288.root
hadd Source file 49: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/c1/e5/H4_v34b_-1GeV_-27.7_004404_20250611T222638Z_004404.root
hadd Source file 50: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/61/b2/H4_v34b_-1GeV_-27.7_004418_20250611T195225Z_004418.root
hadd Source file 51: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/8f/87/H4_v34b_-1GeV_-27.7_004525_20250611T212728Z_004525.root
hadd Source file 52: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/95/20/H4_v34b_-1GeV_-27.7_004676_20250611T200640Z_004676.root
hadd Source file 53: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/5c/22/H4_v34b_-1GeV_-27.7_004784_20250612T075712Z_004784.root
hadd Source file 54: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/95/56/H4_v34b_-1GeV_-27.7_004824_20250612T065326Z_004824.root
hadd Source file 55: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/9d/70/H4_v34b_-1GeV_-27.7_004879_20250611T224921Z_004879.root
hadd Source file 56: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/c5/7a/H4_v34b_-1GeV_-27.7_004934_20250612T055930Z_004934.root
hadd Source file 57: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/42/3d/H4_v34b_-1GeV_-27.7_004998_20250612T060912Z_004998.root
hadd Source file 58: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/93/f0/H4_v34b_-1GeV_-27.7_005031_20250612T061736Z_005031.root
hadd Source file 59: root://mover.pp.rl.ac.uk:1094/pnfs/pp.rl.ac.uk/data/dune/ehn1-beam-np04/fe/1a/H4_v34b_-1GeV_-27.7_005198_20250612T073243Z_005198.root
hadd Source file 60: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/c0/25/H4_v34b_-1GeV_-27.7_005231_20250612T044004Z_005231.root
hadd Source file 61: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/90/3b/H4_v34b_-1GeV_-27.7_005281_20250612T065932Z_005281.root
hadd Source file 62: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/db/60/H4_v34b_-1GeV_-27.7_005297_20250612T044714Z_005297.root
hadd Source file 63: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/04/4b/H4_v34b_-1GeV_-27.7_005477_20250612T045426Z_005477.root
hadd Source file 64: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/a6/9e/H4_v34b_-1GeV_-27.7_005498_20250612T050544Z_005498.root
hadd Source file 65: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/9f/5a/H4_v34b_-1GeV_-27.7_005642_20250612T071707Z_005642.root
hadd Source file 66: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/55/43/H4_v34b_-1GeV_-27.7_005855_20250612T043542Z_005855.root
hadd Source file 67: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/83/aa/H4_v34b_-1GeV_-27.7_005919_20250612T100600Z_005919.root
hadd Source file 68: root://cephc02.gla.scotgrid.ac.uk:1094//cephfs/dune/RSE/ehn1-beam-np04/01/bc/H4_v34b_-1GeV_-27.7_005924_20250612T100400Z_005924.root
hadd Source file 69: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/70/5e/H4_v34b_-1GeV_-27.7_006154_20250612T085218Z_006154.root
hadd Source file 70: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/9a/3d/H4_v34b_-1GeV_-27.7_006188_20250612T102527Z_006188.root
hadd Source file 71: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/dd/0b/H4_v34b_-1GeV_-27.7_006308_20250612T094009Z_006308.root
hadd Source file 72: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/84/93/H4_v34b_-1GeV_-27.7_006321_20250612T013047Z_006321.root
hadd Source file 73: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/7c/46/H4_v34b_-1GeV_-27.7_006527_20250612T014000Z_006527.root
hadd Source file 74: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/b1/34/H4_v34b_-1GeV_-27.7_006593_20250612T083538Z_006593.root
hadd Source file 75: root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/ehn1-beam-np04/32/0d/H4_v34b_-1GeV_-27.7_006599_20250612T111102Z_006599.root
hadd Source file 76: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/bf/1a/H4_v34b_-1GeV_-27.7_006702_20250612T101604Z_006702.root
hadd Source file 77: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/0d/80/H4_v34b_-1GeV_-27.7_006711_20250612T113805Z_006711.root
hadd Source file 78: root://cephc02.gla.scotgrid.ac.uk:1094//cephfs/dune/RSE/ehn1-beam-np04/38/8e/H4_v34b_-1GeV_-27.7_006715_20250613T065705Z_006715.root
hadd Source file 79: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/ehn1-beam-np04/b5/1a/H4_v34b_-1GeV_-27.7_006725_20250612T085355Z_006725.root
hadd Source file 80: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/24/4b/H4_v34b_-1GeV_-27.7_006761_20250612T055353Z_006761.root
hadd Source file 81: root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/ehn1-beam-np04/01/02/H4_v34b_-1GeV_-27.7_006772_20250612T051523Z_006772.root
hadd Source file 82: root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/ehn1-beam-np04/c8/cd/H4_v34b_-1GeV_-27.7_006843_20250613T025420Z_006843.root
hadd Source file 83: root://cephc02.gla.scotgrid.ac.uk:1094//cephfs/dune/RSE/ehn1-beam-np04/eb/bd/H4_v34b_-1GeV_-27.7_007077_20250612T112641Z_007077.root
hadd Source file 84: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/ehn1-beam-np04/43/cc/H4_v34b_-1GeV_-27.7_007317_20250612T103700Z_007317.root
hadd Source file 85: root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/c1/16/H4_v34b_-1GeV_-27.7_007350_20250612T044047Z_007350.root
Error in <TNetXNGFile::Open>: [ERROR] Server responded with an error: [3012] Unable to open file /eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/c1/16/H4_v34b_-1GeV_-27.7_007350_20250612T044047Z_007350.root; Bad address
Error in <TFileMerger::AddFile>: cannot open file root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/c1/16/H4_v34b_-1GeV_-27.7_007350_20250612T044047Z_007350.root
hadd exiting due to error in root://eospublic.cern.ch:1094//eos/experiment/neutplatform/protodune/dune/ehn1-beam-np04/c1/16/H4_v34b_-1GeV_-27.7_007350_20250612T044047Z_007350.root
Error in <TNetXNGFile::Close>: [ERROR] Socket timeout
Error in <TNetXNGFile::Close>: [ERROR] Socket timeout
Error in <TNetXNGFile::Close>: [ERROR] Socket timeout
Error in <TNetXNGFile::Close>: [ERROR] Socket timeout
Error in <TNetXNGFile::Close>: [ERROR] Socket timeout
Error in <TNetXNGFile::Close>: [ERROR] Socket timeout
Error in <TNetXNGFile::Close>: [ERROR] Socket timeout
Traceback (most recent call last):
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f//merge_g4bl.py", line 403, in <module>
do_merge(args)
File "/cvmfs/fifeuser4.opensciencegrid.org/sw/dune/5a837a2f9ce0b916d8725ae4ed0b18872c84fe1f//merge_g4bl.py", line 119, in do_merge
raise Exception('Error in hadd')
Exception: Error in hadd
Exiting with error