21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 426399.1@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 426399.1@justin-prod-sched01.dune.hep.ac.uk | |
| Workflow ID | 8067 | |
| Stage ID | 1 | |
| User name | twalton@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 2097152000 (2000 MiB) | |
| Wall seconds limit | 3600 (1 hours) | |
| Submitted time | 2025-07-02 19:34:55 | |
| Site | UK_RAL-Tier1 | |
| Entry | LIGO_UK_RAL_arc_ce03 | |
| Last heartbeat | 2025-07-02 19:43:01 | |
| From worker node | Hostname | dune001-7273959.0-lcg2641.gridpp.rl.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 2097152000 (2000 MiB) | |
| Wall seconds limit | 216000 (60 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-07-02 19:36:42 | |
| Input files | neardet-2x2-lar-charge:binary-0020057-2024_11_03_00_05_53_CET.hdf5 | |
| Jobscript | Exit code | 0 |
| Real time | 3m (203s) | |
| CPU time | 0m (15s = 7%) | |
| Max RSS bytes | 113852416 (108 MiB) | |
| Outputting started | 2025-07-02 19:40:06 | |
| Output files | ||
| Finished | 2025-07-02 19:43:01 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
Submitting justin jobs via the [justin-jobscript] script.
The node working directory /home/workspace
host is dune001-7273959.0-lcg2641.gridpp.rl.ac.uk
justin site is UK_RAL-Tier1
the current directory is /home/workspace
The workspace directory is /home/workspace
Retrieving the file from the path [/home].
The file data identifier (DID) is [neardet-2x2-lar-charge:binary-0020057-2024_11_03_00_05_53_CET.hdf5]
The file physical file name (PFN) is [root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/neardet-2x2-lar-charge/5d/d3/binary-0020057-2024_11_03_00_05_53_CET.hdf5]
The file Rucio storage element (RSE) is [DUNE_US_FNAL_DISK_STAGE]
The input file is binary-0020057-2024_11_03_00_05_53_CET.hdf5
Using rucio to download file [neardet-2x2-lar-charge:binary-0020057-2024_11_03_00_05_53_CET.hdf5]
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
[32;1m2025-07-02 19:37:47,824 INFO Processing 1 item(s) for input[0m
[31;1m2025-07-02 19:38:44,588 ERROR ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))[0m
[32;1m2025-07-02 19:38:46,119 INFO No preferred protocol impl in rucio.cfg: No section: 'download'[0m
[32;1m2025-07-02 19:38:46,121 INFO Using main thread to download 1 file(s)[0m
[32;1m2025-07-02 19:38:46,122 INFO Preparing download of neardet-2x2-lar-charge:binary-0020057-2024_11_03_00_05_53_CET.hdf5[0m
[32;1m2025-07-02 19:38:46,749 INFO Trying to download with root and timeout of 748s from DUNE_US_FNAL_DISK_STAGE: neardet-2x2-lar-charge:binary-0020057-2024_11_03_00_05_53_CET.hdf5 [0m
[32;1m2025-07-02 19:38:46,955 INFO Using PFN: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/neardet-2x2-lar-charge/5d/d3/binary-0020057-2024_11_03_00_05_53_CET.hdf5[0m
error in request of checking file QoS: is not a valid HTTP or Webdav URL
(process:997): GLib-WARNING **: 19:38:46.958: GError set over the top of a previous GError or uninitialized memory.
This indicates a bug in someone's code. You must ensure an error is NULL before it's set.
The overwriting error message was: \x01 is not a valid HTTP or Webdav URL
/cvmfs/dune.opensciencegrid.org/products/dune/rucio/v37_1_0_post1/NULL/lib/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'dune-rucio.fnal.gov'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
[32;1m2025-07-02 19:39:57,405 INFO File neardet-2x2-lar-charge:binary-0020057-2024_11_03_00_05_53_CET.hdf5 successfully downloaded. 344.147 MB in 8.92 seconds = 38.58 MBps[0m
----------------------------------
Download summary
----------------------------------------
DID neardet-2x2-lar-charge:binary-0020057-2024_11_03_00_05_53_CET.hdf5
Total files (DID): 1
Total files (filtered): 1
Downloaded files: 1
Files already found locally: 0
Files that cannot be downloaded: 0
Downloading the matching light files for the charge+light combination workflow.
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
Running the command [ python /cvmfs/dune.opensciencegrid.org/dunend/2x2//releases/v1.5.0/ndlar_prod_scripts/ND_Production/scripts/GetInputList.py --file=neardet-2x2-lar-charge:binary-0020057-2024_11_03_00_05_53_CET.hdf5 ].
Enter getting the input list of light files for the 2x2 production jobs
Charge file is [neardet-2x2-lar-charge:binary-0020057-2024_11_03_00_05_53_CET.hdf5]
The matching runs/subruns list : []
Exit getting the input list of matching light files.
The number of files is [0].
FATAL::The detector configuration [fsd] is not implemented. Cannot continue.