justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 206976.0@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID206976.0@justin-prod-sched02.dune.hep.ac.uk
Workflow ID7046
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-05-20 18:05:08
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce03
Last heartbeat2025-05-22 20:33:54
From worker nodeHostnamedune001-6478555.0-lcg2708.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6815744000 (6500 MiB)
Wall seconds limit216000 (60 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-05-20 18:06:24
Input filesmonte-carlo-007046-000099
JobscriptExit code0
Real time2d (179547s)
CPU time9d (798597s = 444%)
Max RSS bytes12227330048 (11660 MiB)
Outputting started2025-05-22 19:58:51
Output filesusertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen.root
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen.log
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen.err
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1.root
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1.log
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1.err
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2.root
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2.log
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2.err
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim.root
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim.log
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim.err
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim_reco.root
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim_reco.log
usertests:prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim_reco.err
Finished2025-05-22 20:33:54
Saved logsjustin-logs:206976.0-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

3/protodunevd_v5_ggd_May7PDfix_CRT.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16460 nodes/ 2469 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
22-May-2025 19:53:24 UTC  Initiating request to open input file "prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim.root"
22-May-2025 19:53:24 UTC  Opened input file "prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 37472 traces, 37472 summary
Retagger: tagging trace set: gauss with 30747 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 30747 traces tagged "gauss"
FrameSaver: q=7.25512e+06 n=608351 tag=gauss
wclsFrameSaver: saving 37472 traces tagged "wiener"
FrameSaver: q=7.4722e+06 n=583919 tag=wiener
22-May-2025 19:53:56 UTC  Opened output file with pattern "prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 46209 traces, 46209 summary
Retagger: tagging trace set: gauss with 37301 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 37301 traces tagged "gauss"
FrameSaver: q=8.32065e+06 n=684458 tag=gauss
wclsFrameSaver: saving 46209 traces tagged "wiener"
FrameSaver: q=8.60035e+06 n=644847 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 40232 traces, 40232 summary
Retagger: tagging trace set: gauss with 31316 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 31316 traces tagged "gauss"
FrameSaver: q=7.251e+06 n=575274 tag=gauss
wclsFrameSaver: saving 40232 traces tagged "wiener"
FrameSaver: q=7.59338e+06 n=537885 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48554 traces, 48554 summary
Retagger: tagging trace set: gauss with 38181 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38181 traces tagged "gauss"
FrameSaver: q=9.1561e+06 n=728272 tag=gauss
wclsFrameSaver: saving 48554 traces tagged "wiener"
FrameSaver: q=9.53602e+06 n=688967 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 42567 traces, 42567 summary
Retagger: tagging trace set: gauss with 33955 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 33955 traces tagged "gauss"
FrameSaver: q=7.50894e+06 n=598455 tag=gauss
wclsFrameSaver: saving 42567 traces tagged "wiener"
FrameSaver: q=7.81849e+06 n=560073 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 52689 traces, 52689 summary
Retagger: tagging trace set: gauss with 42129 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 42129 traces tagged "gauss"
FrameSaver: q=9.21577e+06 n=739778 tag=gauss
wclsFrameSaver: saving 52689 traces tagged "wiener"
FrameSaver: q=9.63185e+06 n=694001 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 48780 traces, 48780 summary
Retagger: tagging trace set: gauss with 38331 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 38331 traces tagged "gauss"
FrameSaver: q=9.683e+06 n=716073 tag=gauss
wclsFrameSaver: saving 48780 traces tagged "wiener"
FrameSaver: q=1.01093e+07 n=668189 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 42974 traces, 42974 summary
Retagger: tagging trace set: gauss with 34929 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 34929 traces tagged "gauss"
FrameSaver: q=7.52887e+06 n=608386 tag=gauss
wclsFrameSaver: saving 42974 traces tagged "wiener"
FrameSaver: q=7.84203e+06 n=574724 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 40146 traces, 40146 summary
Retagger: tagging trace set: gauss with 31981 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 31981 traces tagged "gauss"
FrameSaver: q=7.33039e+06 n=609811 tag=gauss
wclsFrameSaver: saving 40146 traces tagged "wiener"
FrameSaver: q=7.58179e+06 n=575587 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 36392 traces, 36392 summary
Retagger: tagging trace set: gauss with 29094 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 29094 traces tagged "gauss"
FrameSaver: q=6.46291e+06 n=514147 tag=gauss
wclsFrameSaver: saving 36392 traces tagged "wiener"
FrameSaver: q=6.73172e+06 n=480374 tag=wiener
22-May-2025 19:58:50 UTC  Closed output file "prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim_reco.root"
22-May-2025 19:58:50 UTC  Closed input file "prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim.root"

=================================================================================================================================
TimeTracker printout (sec)                         Min           Avg           Max         Median          RMS         nEvts   
=================================================================================================================================
Full event                                       27.0251       29.4433       32.5473       28.8171       1.89949        10     
---------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.000375454    0.0013806    0.00199179    0.00141639    0.000531746      10     
reco:rns:RandomNumberSaver                     3.2449e-05    5.35925e-05   0.000210777   3.70305e-05   5.24435e-05      10     
reco:ophit:OpHitFinder                          0.0571949     0.0695182     0.0782934     0.0684294    0.00724611       10     
reco:opflash:OpFlashFinder                      0.0123572     0.0175568     0.0218517     0.0177014    0.00324241       10     
reco:opslicer:OpSlicer                          0.0242471     0.0317854     0.0366725     0.0321943    0.00389782       10     
reco:wclsdatavd:WireCellToolkit                  20.4998       21.5481       22.8307       21.5319      0.751463        10     
reco:gaushit:GausHitFinder                      0.260454      0.369326      0.460116      0.373672      0.0628618       10     
reco:reco3d:SpacePointSolver                     2.55892       3.23459       4.4428        3.16964      0.555926        10     
reco:hitpdune:DisambigFromSpacePoints           0.0232885     0.0303187     0.0434782     0.0292785     0.0060972       10     
reco:pandora:StandardPandora                     2.05545       3.30917       5.47471       2.76196       1.09825        10     
reco:pandoraTrack:LArPandoraTrackCreation        0.12322      0.172405      0.229105      0.170013      0.0347397       10     
reco:pandoraShower:LArPandoraShowerCreation     0.161138      0.245561      0.373933      0.242305      0.0627137       10     
reco:pandoraStdcalo:Calorimetry                 0.0677477     0.0958896     0.121855      0.096692      0.0145849       10     
reco:pandoraGnocalo:GnocchiCalorimetry         0.00995935     0.0124795     0.0164308     0.0122497    0.00172503       10     
[art]:TriggerResults:TriggerResultInserter     1.0566e-05    1.49463e-05   3.4842e-05    1.2549e-05    6.88376e-06      10     
end_path:out1:RootOutput                        2.674e-06    7.0596e-06    1.4171e-05     7.476e-06    3.33949e-06      10     
end_path:out1:RootOutput(write)                 0.243282      0.297506      0.358205      0.292065      0.031579        10     
=================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 6105.49 MB
  Peak resident set size usage (VmHWM): 3918.47 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_cosmics_protodunevd_1_206976_0_1747764395_gen_g4_stage1_g4_stage2_detsim_reco.root', 'start_time': 1747943586.4337928, 'end_time': 1747943930.9869497}
justIN time: 2025-05-22 23:49:48 UTC       justIN version: 01.03.01