Jobsub ID 88038.24@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 88038.24@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3854 |
| Stage ID | 1 |
| User name | epennacc@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 6290407424 (5999 MiB) |
| Wall seconds limit | 80000 (22 hours) |
| Submitted time | 2024-10-28 10:36:05 |
| Site | CA_Victoria |
| Entry | DUNE_CA_Victoria_dune-condor_whole |
| Last heartbeat | 2024-10-28 14:07:49 |
| From worker node | Hostname | dune--heprc--3456932580--74364938604567-1 |
| cpuinfo | Intel Xeon Processor (Skylake, IBRS) |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 6290407424 (5999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-10-28 10:46:23 |
| Input files | monte-carlo-003854-000132
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-10-28 14:07:49 |
| Saved logs | justin-logs:88038.24-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
he 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:32.744] W [sigproc ] <OmnibusSigProc:crm65sigproc65> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:32.875] W [sigproc ] <OmnibusSigProc:crm66sigproc66> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:32.994] W [sigproc ] <OmnibusSigProc:crm67sigproc67> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.092] W [sigproc ] <OmnibusSigProc:crm68sigproc68> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.188] W [sigproc ] <OmnibusSigProc:crm69sigproc69> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.265] W [sigproc ] <OmnibusSigProc:crm70sigproc70> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.382] W [sigproc ] <OmnibusSigProc:crm71sigproc71> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.564] W [sigproc ] <OmnibusSigProc:crm72sigproc72> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.637] W [sigproc ] <OmnibusSigProc:crm73sigproc73> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.705] W [sigproc ] <OmnibusSigProc:crm74sigproc74> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.821] W [sigproc ] <OmnibusSigProc:crm75sigproc75> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.941] W [sigproc ] <OmnibusSigProc:crm76sigproc76> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:33.998] W [sigproc ] <OmnibusSigProc:crm77sigproc77> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:34.113] W [sigproc ] <OmnibusSigProc:crm78sigproc78> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:34.311] W [sigproc ] <OmnibusSigProc:crm79sigproc79> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:34.584] W [sigproc ] <OmnibusSigProc:crm80sigproc80> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:34.906] W [sigproc ] <OmnibusSigProc:crm81sigproc81> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.145] W [sigproc ] <OmnibusSigProc:crm82sigproc82> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.241] W [sigproc ] <OmnibusSigProc:crm83sigproc83> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.283] W [sigproc ] <OmnibusSigProc:crm84sigproc84> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.352] W [sigproc ] <OmnibusSigProc:crm85sigproc85> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.395] W [sigproc ] <OmnibusSigProc:crm86sigproc86> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.432] W [sigproc ] <OmnibusSigProc:crm87sigproc87> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.545] W [sigproc ] <OmnibusSigProc:crm88sigproc88> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.569] W [sigproc ] <OmnibusSigProc:crm89sigproc89> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.596] W [sigproc ] <OmnibusSigProc:crm90sigproc90> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.631] W [sigproc ] <OmnibusSigProc:crm91sigproc91> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.704] W [sigproc ] <OmnibusSigProc:crm92sigproc92> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.770] W [sigproc ] <OmnibusSigProc:crm93sigproc93> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.808] W [sigproc ] <OmnibusSigProc:crm94sigproc94> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.882] W [sigproc ] <OmnibusSigProc:crm95sigproc95> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:35.946] W [sigproc ] <OmnibusSigProc:crm96sigproc96> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.019] W [sigproc ] <OmnibusSigProc:crm97sigproc97> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.077] W [sigproc ] <OmnibusSigProc:crm98sigproc98> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.127] W [sigproc ] <OmnibusSigProc:crm99sigproc99> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.174] W [sigproc ] <OmnibusSigProc:crm100sigproc100> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.242] W [sigproc ] <OmnibusSigProc:crm101sigproc101> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.308] W [sigproc ] <OmnibusSigProc:crm102sigproc102> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.379] W [sigproc ] <OmnibusSigProc:crm103sigproc103> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.447] W [sigproc ] <OmnibusSigProc:crm104sigproc104> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.472] W [sigproc ] <OmnibusSigProc:crm105sigproc105> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.667] W [sigproc ] <OmnibusSigProc:crm106sigproc106> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.726] W [sigproc ] <OmnibusSigProc:crm107sigproc107> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.826] W [sigproc ] <OmnibusSigProc:crm108sigproc108> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.849] W [sigproc ] <OmnibusSigProc:crm109sigproc109> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.891] W [sigproc ] <OmnibusSigProc:crm110sigproc110> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
[13:58:36.953] W [sigproc ] <OmnibusSigProc:crm111sigproc111> The 'wiener_threshold_tag' is obsolete, thresholds in summary on 'wiener' tagged traces
Inputer: "wclsSimDepoSource:"
Outputer: "wclsSimChannelSink:postdrift"
Outputer: "wclsFrameSaver:spsignals"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
wclsFrameSaver: promising to produce recob::Wires named "dnnsp"
%MSG-s ArtException: TriggerResultInserter:TriggerResults@Construction 28-Oct-2024 14:00:52 UTC ModuleConstruction
cet::exception caught in art
---- CatalogServiceError BEGIN
Input file not found: prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241028T104759Z_gen_000132_supernova_g4stage1_g4stage2.root.
---- CatalogServiceError END
%MSG
Art has completed and will exit with status 24.
ERROR: lar (detsim) exit code: 31024
output files size:
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241028T104759Z_gen_000132.root 85448644
../justin-jobscript: line 168: return: can only `return' from a function or sourced script
============reco =========================
../justin-jobscript: line 180: 1394 Killed lar -c standard_reco1_dunevd10kt_1x8x14_3view_30deg.fcl ${detsim_name}.root -o ${reco_name}.root
ERROR: lar (reco) exit code: 51137
output files size:
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241028T104759Z_gen_000132.root 85448644
../justin-jobscript: line 194: return: can only `return' from a function or sourced script
Traceback (most recent call last):
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_91_04d00/bin/extractor_prod.py", line 434, in <module>
main()
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_91_04d00/bin/extractor_prod.py", line 373, in main
mddict = expSpecificMetadata.getmetadata()
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_91_04d00/bin/extractor_prod.py", line 344, in getmetadata
jobt = self.get_job(proc)
File "/cvmfs/dune.opensciencegrid.org/products/dune/duneutil/v09_91_04d00/bin/extractor_prod.py", line 69, in get_job
raise RuntimeError('sam_metadata_dumper returned nonzero exit status {}.'.format(rc))
RuntimeError: sam_metadata_dumper returned nonzero exit status 1.
python: can't open file '/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/9227c25e1635d18779282cfab801fcdc7cd063b1/pdjson2metadata': [Errno 2] No such file or directory
ERROR: metadata writing 52002
output files size:
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241028T104759Z_gen_000132.root 85448644
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241028T104759Z_gen_000132_supernova_g4stage1_g4stage2_detsim_reco.root.ext.json 0
prodbackground_radiological_decay0_dunevd10kt_1x8x14_20241028T104759Z_gen_000132_supernova_g4stage1_g4stage2_detsim_reco.root.json 0
../justin-jobscript: line 232: return: can only `return' from a function or sourced script
Exiting with error
../justin-jobscript: line 240: return: can only `return' from a function or sourced script 