21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 233952.0@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 233952.0@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 8064 | |
| Stage ID | 1 | |
| User name | twalton@fnal.gov | |
| HTCondor Group | group_dune.prod_mcsim | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 2097152000 (2000 MiB) | |
| Wall seconds limit | 3600 (1 hours) | |
| Submitted time | 2025-07-02 17:32:44 | |
| Site | US_SU-ITS | |
| Entry | Glow_US_Syracuse_condor-ce2 | |
| Last heartbeat | 2025-07-02 17:41:10 | |
| From worker node | Hostname | CRUSH-OSG-C7-10-5-198-201 |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2698 v3 @ 2.30GHz | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 2097152000 (2000 MiB) | |
| Wall seconds limit | 171000 (47 hours) | |
| GPU | ||
| Inner Apptainer? | False | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-07-02 17:35:05 | |
| Input files | neardet-2x2-lar-charge:binary-0050011-2024_07_07_13_59_33_CDT.hdf5 | |
| Jobscript | Exit code | 0 |
| Real time | 0m (0s) | |
| CPU time | 0m (0s = 0%) | |
| Max RSS bytes | 0 (0 MiB) | |
| Outputting started | 2025-07-02 17:37:29 | |
| Output files | ||
| Finished | 2025-07-02 17:41:10 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
Submitting justin jobs via the [justin-jobscript] script. Setup and enter the conda environment for the software release [v1.5.0] The node working directory /srv/home/workspace host is CRUSH-OSG-C7-10-5-198-201 justin site is US_SU-ITS the current directory is /srv/home/workspace The workspace directory is /srv/home/workspace Retrieving the file from the path [/srv/home]. The file data identifier (DID) is [neardet-2x2-lar-charge:binary-0050011-2024_07_07_13_59_33_CDT.hdf5] The file physical file name (PFN) is [root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/neardet-2x2-lar-charge/87/32/binary-0050011-2024_07_07_13_59_33_CDT.hdf5] The file Rucio storage element (RSE) is [DUNE_US_FNAL_DISK_STAGE] The input file is binary-0050011-2024_07_07_13_59_33_CDT.hdf5 Using rucio to download file [neardet-2x2-lar-charge:binary-0050011-2024_07_07_13_59_33_CDT.hdf5] Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/ [32;1m2025-07-02 17:35:47,226 INFO Processing 1 item(s) for input[0m [33;1m2025-07-02 17:36:47,691 WARNING Waiting 0.25s due to reason: server returned 504 [0m [32;1m2025-07-02 17:36:48,385 INFO No preferred protocol impl in rucio.cfg: No section: 'download'[0m [32;1m2025-07-02 17:36:48,385 INFO Using main thread to download 1 file(s)[0m [32;1m2025-07-02 17:36:48,386 INFO Preparing download of neardet-2x2-lar-charge:binary-0050011-2024_07_07_13_59_33_CDT.hdf5[0m [32;1m2025-07-02 17:36:48,577 INFO Trying to download with root and timeout of 547s from DUNE_US_FNAL_DISK_STAGE: neardet-2x2-lar-charge:binary-0050011-2024_07_07_13_59_33_CDT.hdf5 [0m [32;1m2025-07-02 17:36:49,128 INFO Using PFN: root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/neardet-2x2-lar-charge/87/32/binary-0050011-2024_07_07_13_59_33_CDT.hdf5[0m error in request of checking file QoS: is not a valid HTTP or Webdav URL (process:938): GLib-WARNING **: 17:36:49.130: GError set over the top of a previous GError or uninitialized memory. This indicates a bug in someone's code. You must ensure an error is NULL before it's set. The overwriting error message was: \x01 is not a valid HTTP or Webdav URL /cvmfs/dune.opensciencegrid.org/products/dune/rucio/v37_1_0_post1/NULL/lib/python3.9/site-packages/urllib3/connectionpool.py:1061: InsecureRequestWarning: Unverified HTTPS request is being made to host 'dune-rucio.fnal.gov'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings warnings.warn( [32;1m2025-07-02 17:37:23,992 INFO File neardet-2x2-lar-charge:binary-0050011-2024_07_07_13_59_33_CDT.hdf5 successfully downloaded. 243.555 MB in 34.33 seconds = 7.09 MBps[0m ---------------------------------- Download summary ---------------------------------------- DID neardet-2x2-lar-charge:binary-0050011-2024_07_07_13_59_33_CDT.hdf5 Total files (DID): 1 Total files (filtered): 1 Downloaded files: 1 Files already found locally: 0 Files that cannot be downloaded: 0 Downloading the matching light files for the charge+light combination workflow. Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/ Running the command [ python /cvmfs/dune.opensciencegrid.org/dunend/2x2//releases/v1.5.0/ndlar_prod_scripts/ND_Production/scripts/GetInputList.py --file=neardet-2x2-lar-charge:binary-0050011-2024_07_07_13_59_33_CDT.hdf5 ]. python: can't open file '/cvmfs/dune.opensciencegrid.org/dunend/2x2//releases/v1.5.0/ndlar_prodscripts/ND_Production/scripts/GetInputList.py': [Errno 2] No such file or directory Failed to get the matching light files.