Jobsub ID 398484.177@justin-prod-sched01.dune.hep.ac.uk
Jobsub ID | 398484.177@justin-prod-sched01.dune.hep.ac.uk |
Workflow ID | 6960 |
Stage ID | 1 |
User name | calcuttj@fnal.gov |
HTCondor Group | group_dune |
Requested | Processors | 1 |
GPU | No |
RSS bytes | 8387559424 (7999 MiB) |
Wall seconds limit | 80000 (22 hours) |
Submitted time | 2025-05-15 06:43:03 |
Site | CERN |
Entry | CMSHTPC_T2_CH_CERN_ce515 |
Last heartbeat | 2025-05-15 10:01:05 |
From worker node | Hostname | b9p31p4711.cern.ch |
cpuinfo | AMD EPYC 7302 16-Core Processor |
OS release | Scientific Linux release 7.9 (Nitrogen) |
Processors | 1 |
RSS bytes | 8387559424 (7999 MiB) |
Wall seconds limit | 343800 (95 hours) |
GPU | |
Inner Apptainer? | True |
Job state | outputting_failed |
Allocator name | justin-allocator-pro.dune.hep.ac.uk |
Started | 2025-05-15 07:33:18 |
Input files | monte-carlo-006960-004118
|
Jobscript | Exit code | 0 |
Real time | 2h (8845s) |
CPU time | 2h (8563s = 96%) |
Max RSS bytes | 12295876608 (11726 MiB) |
Outputting started | 2025-05-15 10:00:44 |
Output files | |
Finished | 2025-05-15 10:01:05 |
List job events Wrapper job log |
Jobscript log (last 10,000 characters)
ActionsMap_
registering to trackingActionsMap_
registering to steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
15-May-2025 11:55:00 CEST Initiating request to open input file "placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5.root"
15-May-2025 11:55:01 CEST Opened input file "placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5.root"
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 90257 traces, 90257 summary
Retagger: tagging trace set: gauss with 74289 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 74289 traces tagged "gauss"
FrameSaver: q=2.05025e+06 n=1116812 tag=gauss
wclsFrameSaver: saving 90257 traces tagged "wiener"
FrameSaver: q=2.29348e+06 n=1032542 tag=wiener
15-May-2025 11:55:47 CEST Opened output file with pattern "placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5_stage6.root"
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 110704 traces, 110704 summary
Retagger: tagging trace set: gauss with 90639 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 90639 traces tagged "gauss"
FrameSaver: q=2.99378e+06 n=1374516 tag=gauss
wclsFrameSaver: saving 110704 traces tagged "wiener"
FrameSaver: q=3.32359e+06 n=1246500 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 75287 traces, 75287 summary
Retagger: tagging trace set: gauss with 64634 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 64634 traces tagged "gauss"
FrameSaver: q=1.44857e+06 n=895676 tag=gauss
wclsFrameSaver: saving 75287 traces tagged "wiener"
FrameSaver: q=1.62679e+06 n=836436 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 82129 traces, 82129 summary
Retagger: tagging trace set: gauss with 69878 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 69878 traces tagged "gauss"
FrameSaver: q=1.79044e+06 n=1022629 tag=gauss
wclsFrameSaver: saving 82129 traces tagged "wiener"
FrameSaver: q=1.98275e+06 n=937885 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 67727 traces, 67727 summary
Retagger: tagging trace set: gauss with 58836 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 58836 traces tagged "gauss"
FrameSaver: q=1.25672e+06 n=825291 tag=gauss
wclsFrameSaver: saving 67727 traces tagged "wiener"
FrameSaver: q=1.39563e+06 n=775273 tag=wiener
15-May-2025 11:59:24 CEST Closed output file "placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5_stage6.root"
15-May-2025 11:59:24 CEST Closed input file "placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5.root"
================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
================================================================================================================================
Full event 43.1952 46.2674 49.4113 45.663 2.52531 5
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000685182 0.00123068 0.00171149 0.00121846 0.000334028 5
simulate:wclsnfspbadapa1:WireCellToolkit 42.8854 45.891 49.0791 45.1738 2.52354 5
[art]:TriggerResults:TriggerResultInserter 3.847e-05 6.00812e-05 9.9182e-05 4.7851e-05 2.32403e-05 5
end_path:out1:RootOutput 6.65e-06 1.41522e-05 3.0291e-05 1.05e-05 8.34937e-06 5
end_path:out1:RootOutput(write) 0.305458 0.371791 0.484967 0.361948 0.0620527 5
================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4970.74 MB
Peak resident set size usage (VmHWM): 3331.93 MB
Details saved in: 'mem.db'
====================================================================================================
{'art_out': 'placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5_stage6.root'}
extract
Running extract
['lar', '-c', 'run_extract_wire.fcl', 'placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5_stage6.root', '-n', '-1']
Stdout:
Begin processing the 1st record. run: 1 subRun: 0 event: 1 at 15-May-2025 11:59:56 CEST
Begin processing the 2nd record. run: 1 subRun: 0 event: 2 at 15-May-2025 12:00:05 CEST
Begin processing the 3rd record. run: 1 subRun: 0 event: 3 at 15-May-2025 12:00:16 CEST
Begin processing the 4th record. run: 1 subRun: 0 event: 4 at 15-May-2025 12:00:24 CEST
Begin processing the 5th record. run: 1 subRun: 0 event: 5 at 15-May-2025 12:00:34 CEST
Art has completed and will exit with status 0.
Stderr:
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/dune.opensciencegrid.org/products/dune/dunecore/v10_01_03d00/gdml/protodunehd_v6_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 1148
Info in <TGeoManager::CloseGeometry>: 25608 nodes/ 5535 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
registering to primaryGeneratorActionsMap_
registering to eventActionsMap_
registering to trackingActionsMap_
registering to steppingActionsMap_
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
15-May-2025 11:59:55 CEST Initiating request to open input file "placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5_stage6.root"
15-May-2025 11:59:56 CEST Opened input file "placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5_stage6.root"
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
initializeNtuple() called with cflag_=6
15-May-2025 12:00:43 CEST Closed input file "placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5_stage6.root"
======================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
======================================================================================================================
Full event 2.13747 4.05501 6.01392 4.35727 1.27502 5
----------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000685052 0.00116383 0.00161031 0.00138456 0.000377989 5
end_path:extractwire:ExtractWire 2.13559 4.05343 6.0123 4.35628 1.27513 5
======================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 3170.97 MB
Peak resident set size usage (VmHWM): 2310.64 MB
Details saved in: 'mem.db'
====================================================================================================
{'art_out': 'placeholder_1_398484_177_1747294410_stage1_stage2_stage3_stage4_stage5_stage6_extract.root', 'start_time': 1747303165.7104325, 'end_time': 1747303243.9314392}
extracted_wires_r00001_s00000.h5