Jobsub ID 204217.40@justin-prod-sched02.dune.hep.ac.uk
Jobsub ID | 204217.40@justin-prod-sched02.dune.hep.ac.uk |
Workflow ID | 6960 |
Stage ID | 1 |
User name | calcuttj@fnal.gov |
HTCondor Group | group_dune |
Requested | Processors | 1 |
GPU | No |
RSS bytes | 8387559424 (7999 MiB) |
Wall seconds limit | 80000 (22 hours) |
Submitted time | 2025-05-14 23:36:58 |
Site | UK_Durham |
Entry | DUNE_UK_SGridDurham_ce3 |
Last heartbeat | 2025-05-15 03:48:53 |
From worker node | Hostname | n255.dur.scotgrid.ac.uk |
cpuinfo | AMD EPYC 7763 64-Core Processor |
OS release | Scientific Linux release 7.9 (Nitrogen) |
Processors | 1 |
RSS bytes | 8387559424 (7999 MiB) |
Wall seconds limit | 171000 (47 hours) |
GPU | |
Inner Apptainer? | True |
Job state | outputting_failed |
Allocator name | justin-allocator-pro.dune.hep.ac.uk |
Started | 2025-05-15 02:00:11 |
Input files | monte-carlo-006960-002494
|
Jobscript | Exit code | 0 |
Real time | 1h (6505s) |
CPU time | 1h (6419s = 98%) |
Max RSS bytes | 12165226496 (11601 MiB) |
Outputting started | 2025-05-15 03:48:37 |
Output files | |
Finished | 2025-05-15 03:48:53 |
List job events Wrapper job log |
Jobscript log (last 10,000 characters)
ering to eventActionsMap_
registering to trackingActionsMap_
registering to steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
15-May-2025 04:43:33 BST Initiating request to open input file "placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5.root"
15-May-2025 04:43:34 BST Opened input file "placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5.root"
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 105900 traces, 105900 summary
Retagger: tagging trace set: gauss with 88967 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 88967 traces tagged "gauss"
FrameSaver: q=2.72216e+06 n=1431818 tag=gauss
wclsFrameSaver: saving 105900 traces tagged "wiener"
FrameSaver: q=3.01366e+06 n=1314129 tag=wiener
15-May-2025 04:44:20 BST Opened output file with pattern "placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5_stage6.root"
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 95535 traces, 95535 summary
Retagger: tagging trace set: gauss with 75288 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 75288 traces tagged "gauss"
FrameSaver: q=2.2126e+06 n=1072954 tag=gauss
wclsFrameSaver: saving 95535 traces tagged "wiener"
FrameSaver: q=2.48445e+06 n=968179 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 101602 traces, 101602 summary
Retagger: tagging trace set: gauss with 80978 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 80978 traces tagged "gauss"
FrameSaver: q=2.22938e+06 n=1185203 tag=gauss
wclsFrameSaver: saving 101602 traces tagged "wiener"
FrameSaver: q=2.48891e+06 n=1054207 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 104902 traces, 104902 summary
Retagger: tagging trace set: gauss with 82234 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 82234 traces tagged "gauss"
FrameSaver: q=2.32761e+06 n=1173622 tag=gauss
wclsFrameSaver: saving 104902 traces tagged "wiener"
FrameSaver: q=2.62331e+06 n=1043070 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 96592 traces, 96592 summary
Retagger: tagging trace set: gauss with 78760 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 78760 traces tagged "gauss"
FrameSaver: q=2.21714e+06 n=1178093 tag=gauss
wclsFrameSaver: saving 96592 traces tagged "wiener"
FrameSaver: q=2.46353e+06 n=1072440 tag=wiener
15-May-2025 04:47:29 BST Closed output file "placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5_stage6.root"
15-May-2025 04:47:29 BST Closed input file "placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 40.9086 41.9252 43.7187 41.5151 1.04076 5
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000895859 0.00124967 0.00168903 0.00122109 0.000262238 5
simulate:wclsnfspbadapa1:WireCellToolkit 40.5791 41.5565 43.171 41.2071 0.967763 5
[art]:TriggerResults:TriggerResultInserter 3.4926e-05 6.20436e-05 0.000149744 4.4664e-05 4.40441e-05 5
end_path:out1:RootOutput 5.531e-06 1.99158e-05 6.6676e-05 8.937e-06 2.3428e-05 5
end_path:out1:RootOutput(write) 0.300219 0.359364 0.532581 0.322422 0.087101 5
================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4895.55 MB
Peak resident set size usage (VmHWM): 3270.34 MB
Details saved in: 'mem.db'
====================================================================================================
{'art_out': 'placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5_stage6.root'}
extract
Running extract
['lar', '-c', 'run_extract_wire.fcl', 'placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5_stage6.root', '-n', '-1']
Stdout:
Begin processing the 1st record. run: 1 subRun: 0 event: 1 at 15-May-2025 04:47:56 BST
Begin processing the 2nd record. run: 1 subRun: 0 event: 2 at 15-May-2025 04:48:04 BST
Begin processing the 3rd record. run: 1 subRun: 0 event: 3 at 15-May-2025 04:48:11 BST
Begin processing the 4th record. run: 1 subRun: 0 event: 4 at 15-May-2025 04:48:19 BST
Begin processing the 5th record. run: 1 subRun: 0 event: 5 at 15-May-2025 04:48:28 BST
Art has completed and will exit with status 0.
Stderr:
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_01_03d00/gdml/protodunehd_v6_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 25608 nodes/ 5535 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
registering to primaryGeneratorActionsMap_
registering to eventActionsMap_
registering to trackingActionsMap_
registering to steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
15-May-2025 04:47:55 BST Initiating request to open input file "placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5_stage6.root"
15-May-2025 04:47:56 BST Opened input file "placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5_stage6.root"
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
15-May-2025 04:48:36 BST Closed input file "placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5_stage6.root"
======================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
======================================================================================================================
Full event 4.12777 5.18806 6.90114 5.14979 0.949869 5
----------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00076472 0.00131435 0.00198016 0.00127381 0.000447845 5
end_path:extractwire:ExtractWire 4.12676 5.18634 6.89919 5.14802 0.949708 5
======================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3128.27 MB
Peak resident set size usage (VmHWM): 2269.1 MB
Details saved in: 'mem.db'
====================================================================================================
{'art_out': 'placeholder_1_204217_40_1747274419_stage1_stage2_stage3_stage4_stage5_stage6_extract.root', 'start_time': 1747280849.9117978, 'end_time': 1747280916.9198673}
extracted_wires_r00001_s00000.h5