Jobsub ID 75146.0@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	75146.0@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	3314
Stage ID	1
User name	amoor@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-09-20 08:53:17
Site	US_UConn-HPC
Entry	GLUEX_US_UConn-HPC_osgce
Last heartbeat	2024-09-20 08:55:44
From worker node	Hostname	cn411
	cpuinfo	AMD EPYC 7452 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	172800 (48 hours)
	Inner Apptainer?	True
Job state	jobscript_error
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-09-20 08:54:08
Input files	usertests:000343_reco_data_2024-08-16T_162142Z_reco_data_2024-08-19T_110647Z_reco_data_2024-08-20T_095436Z_reco_data_2024-08-22T_091530Z_reco_data_2024-09-16T_135837Z.root
Jobscript	Exit code	1
	Real time	0m (0s)
	CPU time	0m (0s = 0%)
Outputting started
Output files
Finished	2024-09-20 08:55:44
Saved logs	justin-logs:75146.0-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events Wrapper job log

Jobscript log (last 10,000 characters)

AFM ana jobscript.
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/43/b2/000343_reco_data_2024-08-16T_162142Z_reco_data_2024-08-19T_110647Z_reco_data_2024-08-20T_095436Z_reco_data_2024-08-22T_091530Z_reco_data_2024-09-16T_135837Z.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_4_3b/Linux64bit+3.10-2.17-e20-p3913-prof/lib/libXrdPosixPreload.so
:.:./job:/cvmfs/dune.opensciencegrid.org/products/dune/dunesw/v09_75_03d00/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/protoduneana/v09_75_03d00/job:/cvmfs/larsoft.opensciencegrid.org/products/geant4reweight/v01_18_01/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/duneprototypes/v09_75_03d00/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/dunesim/v09_75_03d00/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/dunecalib/v09_75_03d00/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/dunedataprep/v09_75_03d00/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/duneana/v09_75_03d00/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/dunereco/v09_75_03d00/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/duneopdet/v09_75_03d00/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v09_75_03d00/fcl:/cvmfs/larsoft.opensciencegrid.org/products/nusystematics/v01_03_09/fcl:/cvmfs/larsoft.opensciencegrid.org/products/systematicstools/v01_03_01/fcl:.:./job:/cvmfs/larsoft.opensciencegrid.org/products/larwirecell/v09_14_01/fcl:/cvmfs/larsoft.opensciencegrid.org/products/larsimdnn/v09_05_02/job:/cvmfs/larsoft.opensciencegrid.org/products/larrecodnn/v09_21_03/job:/cvmfs/larsoft.opensciencegrid.org/products/larpandora/v09_21_02/job:/cvmfs/larsoft.opensciencegrid.org/products/larexamples/v09_08_02/job:/cvmfs/larsoft.opensciencegrid.org/products/lareventdisplay/v09_10_02/job:/cvmfs/larsoft.opensciencegrid.org/products/nuevdb/v1_08_01/fcl:/cvmfs/larsoft.opensciencegrid.org/products/larana/v09_14_02/job:/cvmfs/larsoft.opensciencegrid.org/products/larreco/v09_20_02/job:/cvmfs/larsoft.opensciencegrid.org/products/larsim/v09_36_02/job:/cvmfs/larsoft.opensciencegrid.org/products/nugen/v1_19_04/fcl:/cvmfs/larsoft.opensciencegrid.org/products/larg4/v09_16_01/fcl:/cvmfs/larsoft.opensciencegrid.org/products/nurandom/v1_10_01/fcl:/cvmfs/larsoft.opensciencegrid.org/products/nug4/v1_15_01/fcl:/cvmfs/larsoft.opensciencegrid.org/products/larevt/v09_09_01/job:/cvmfs/larsoft.opensciencegrid.org/products/lardata/v09_15_01/job:/cvmfs/larsoft.opensciencegrid.org/products/lardataalg/v09_15_01/job:/cvmfs/larsoft.opensciencegrid.org/products/larcore/v09_10_00/job:/cvmfs/larsoft.opensciencegrid.org/products/larcorealg/v09_12_00/job:/cvmfs/larsoft.opensciencegrid.org/products/artg4tk/v11_00_01/fcl:/cvmfs/dune.opensciencegrid.org/products/dune/dunepdlegacy/v1_00_03/fcl
=== Start last 50 lines of lar log file ===
Failed to parse the configuration file 'rsj_standard_ana_dune10kt_1x2x6.fcl' with exception
---- search_path BEGIN
  Can't find file "rsj_standard_ana_dune10kt_1x2x6.fcl"
---- search_path END

Art has completed and will exit with status 90.
=== End last 50 lines of lar log file ===
lar exit code 0
Traceback (most recent call last):
  File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 434, in <module>
    main()
  File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 373, in main
    mddict = expSpecificMetadata.getmetadata()
  File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 344, in getmetadata
    jobt = self.get_job(proc)
  File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_03d00/bin/extractor_prod.py", line 69, in get_job
    raise RuntimeError('sam_metadata_dumper returned nonzero exit status {}.'.format(rc))
RuntimeError: sam_metadata_dumper returned nonzero exit status 1.
extractor_prod.py exit code 1
Error reading metadata from file: Expecting value: line 1 column 1 (char 0)
pdjson2metadata exit code 1
.:
total 16
-rw-r--r-- 1 osgusers domain users 4162 Sep 20 04:54 jobscript.log
-rw-r--r-- 1 osgusers domain users  344 Sep 20 04:54 all-input-dids.txt
-rw-r--r-- 1 osgusers domain users  241 Sep 20 04:54 000343_reco_data_2024-08-16T_162142Z_reco_data_2024-08-19T_110647Z_reco_data_2024-08-20T_095436Z_reco_data_2024-08-22T_091530Z_reco_data_2024-09-16T_135837Z_reco_2024-09-20T_085426Z.log
-rw-r--r-- 1 osgusers domain users    0 Sep 20 04:54 000343_reco_data_2024-08-16T_162142Z_reco_data_2024-08-19T_110647Z_reco_data_2024-08-20T_095436Z_reco_data_2024-08-22T_091530Z_reco_data_2024-09-16T_135837Z_reco_data_2024-09-20T_085426Z.root.ext.json
-rw-r--r-- 1 osgusers domain users    0 Sep 20 04:54 000343_reco_data_2024-08-16T_162142Z_reco_data_2024-08-19T_110647Z_reco_data_2024-08-20T_095436Z_reco_data_2024-08-22T_091530Z_reco_data_2024-09-16T_135837Z_reco_data_2024-09-20T_085426Z.root.json