Jobsub ID 279750.2@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 279750.2@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3815 |
| Stage ID | 1 |
| User name | aralaiko@fnal.gov |
| HTCondor Group | group_dune |
| Requested | Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2024-10-24 19:33:48 |
| Site | UK_Edinburgh |
| Entry | DUNE_UK_SGridECDF_ce1 |
| Last heartbeat | 2024-10-24 20:07:24 |
| From worker node | Hostname | node1g18.ecdf.ed.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4194304000 (4000 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-10-24 19:35:01 |
| Input files | justin-tutorial:tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-10-24 20:07:24 |
| Saved logs | justin-logs:279750.2-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
ier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 164. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 165. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 166. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 167. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 168. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 169. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 170. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 171. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 172. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 173. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 174. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 175. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 176. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 177. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.741490 +0100][Debug ][XRootD ][ 1282] 178. Waited at server request. Resending: root://meitner.tier2.hep.manchester.ac.uk:1094//cephfs/experiments/dune/RSE/justin-tutorial/09/9c/tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712.hdf5
[2024-10-24 21:07:02.742185 +0100][Debug ][ExDbgMsg ][ 1282] [meitner.tier2.hep.manchester.ac.uk:1094] Destroying MsgHandler: 0xb7724c0.
HDF5-DIAG: Error detected in HDF5 (1.12.2) thread 0:
#000: /scratch/workspace/build-single/BUILDTYPE/prof/QUAL/e20/label1/swarm/label2/SLF7/build/hdf5/v1_12_2a/source/hdf5-1.12.2/src/H5F.c line 707 in H5Fclose(): not a file ID
major: Invalid arguments to routine
minor: Inappropriate type
DataPrepByApaModule::endJob: # events processed: 0
DataPrepByApaModule::endJob: # events skipped: 0
====================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
====================================================================================================================
[ No processed events ]
====================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 2725.57 MB
Peak resident set size usage (VmHWM): 1711.81 MB
Details saved in: 'mem.db'
====================================================================================================
PandoraMonitoring, only able to use default TApplication (limited functionality).
PandoraMonitoring::SaveTree, error: No tree with name 'Validation' exists.
ToolBasedRawDigitPrepService:dtor: Event count: 0
ToolBasedRawDigitPrepService:dtor: Call count: 0
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor: digitReader :0.00 sec
ToolBasedRawDigitPrepService:dtor: vdcb_adcChannelRawRmsFiller :0.00 sec
ToolBasedRawDigitPrepService:dtor: adcSampleFiller :0.00 sec
ToolBasedRawDigitPrepService:dtor: vdbcb_adcScaleAdcToKe :0.00 sec
ToolBasedRawDigitPrepService:dtor: vdbcb_cnrw :0.00 sec
ToolBasedRawDigitPrepService:dtor: adcKeepAllSignalFinder :0.00 sec
ToolBasedRawDigitPrepService:dtor: vdbcb_adcScaleKeToAdc :0.00 sec
Art has completed and will exit with status 0.
[2024-10-24 21:07:03.624029 +0100][Debug ][JobMgr ][ 1282] Stopping the job manager...
[2024-10-24 21:07:03.624366 +0100][Debug ][JobMgr ][ 1282] Job manager stopped
[2024-10-24 21:07:03.624515 +0100][Debug ][TaskMgr ][ 1282] Stopping the task manager...
[2024-10-24 21:07:03.624596 +0100][Debug ][TaskMgr ][ 1282] Task manager stopped
[2024-10-24 21:07:03.624739 +0100][Debug ][Poller ][ 1282] Stopping the poller...
[2024-10-24 21:07:03.624874 +0100][Debug ][TaskMgr ][ 1282] Requesting unregistration of: "TickGeneratorTask for: root://meitner.tier2.hep.manchester.ac.uk:1094"
[2024-10-24 21:07:03.624917 +0100][Debug ][AsyncSock ][ 1282] [meitner.tier2.hep.manchester.ac.uk:1094.0] Closing the socket
[2024-10-24 21:07:03.625005 +0100][Debug ][Poller ][ 1282] <[::ffff:192.41.104.205]:44840><--><[::ffff:195.194.108.197]:1094> Removing socket from the poller
[2024-10-24 21:07:03.625411 +0100][Debug ][PostMaster ][ 1282] [meitner.tier2.hep.manchester.ac.uk:1094] Destroying stream
[2024-10-24 21:07:03.625476 +0100][Debug ][AsyncSock ][ 1282] [meitner.tier2.hep.manchester.ac.uk:1094.0] Closing the socket
=== End last 100 lines of lar log file ===
lar exit code 0
Traceback (most recent call last):
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_00d00/bin/extractor_prod.py", line 434, in <module>
main()
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_00d00/bin/extractor_prod.py", line 373, in main
mddict = expSpecificMetadata.getmetadata()
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_00d00/bin/extractor_prod.py", line 344, in getmetadata
jobt = self.get_job(proc)
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_75_00d00/bin/extractor_prod.py", line 69, in get_job
raise RuntimeError('sam_metadata_dumper returned nonzero exit status {}.'.format(rc))
RuntimeError: sam_metadata_dumper returned nonzero exit status 1.
extractor_prod.py exit code 1
Error reading metadata from file: Expecting value: line 1 column 1 (char 0)
pdjson2metadata exit code 1
.:
total 504
-rw-r--r-- 1 gl05pi6 eddie_users 424604 Oct 24 21:07 tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712_reco_2024-10-24T_193504Z.log
-rw-r--r-- 1 gl05pi6 eddie_users 36864 Oct 24 21:07 mem.db
-rw-r--r-- 1 gl05pi6 eddie_users 21072 Oct 24 21:07 jobscript.log
-rw-r--r-- 1 gl05pi6 eddie_users 16384 Oct 24 20:37 time.db
-rw-r--r-- 1 gl05pi6 eddie_users 519 Oct 24 21:07 tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712_reco_hist.root
-rw-r--r-- 1 gl05pi6 eddie_users 182 Oct 24 20:35 all-input-dids.txt
-rw-r--r-- 1 gl05pi6 eddie_users 0 Oct 24 20:37 Pandora_Events.pndr
-rw-r--r-- 1 gl05pi6 eddie_users 0 Oct 24 20:36 debugprod.log
-rw-r--r-- 1 gl05pi6 eddie_users 0 Oct 24 21:07 tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712_reco_data_2024-10-24T_193504Z.root.ext.json
-rw-r--r-- 1 gl05pi6 eddie_users 0 Oct 24 21:07 tut_np02bde_307160012_np02_bde_coldbox_run012352_0055_20211216T000712_reco_data_2024-10-24T_193504Z.root.json 