justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 167106.42@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID167106.42@justin-prod-sched02.dune.hep.ac.uk
Workflow ID5780
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-03-28 15:52:21
SiteCA_SFU
EntryDUNE_CA_SFU_lcg-ce3
Last heartbeat2025-03-28 17:06:11
From worker nodeHostnamecdr939.int.cedar.computecanada.ca
cpuinfoIntel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6815744000 (6500 MiB)
Wall seconds limit84598 (23 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-03-28 16:45:57
Input filesmonte-carlo-005780-000792
JobscriptExit code0
Real time19m (1160s)
CPU time18m (1137s = 98%)
Max RSS bytes1730666496 (1650 MiB)
Outputting started2025-03-28 17:05:18
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim_reco.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim_reco.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05780/1/014/prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim_reco.err
Finished2025-03-28 17:06:11
Saved logsjustin-logs:167106.42-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

tion...
Begin: 1.68635e+08
0 1.68147e+08
1 1.68147e+08
Now with regularization...
Begin: 1.67932e+08
0 1.67932e+08
Art has completed and will exit with status 0.

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  pandoraWriter
  pandoracalipid
  pandorapid
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/497fe0b9af7ebfe5ac11fff59041e9416f4d09cb/protodunevd_v4_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16499 nodes/ 2408 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
28-Mar-2025 09:55:55 PDT  Initiating request to open input file "prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim.root"
28-Mar-2025 09:55:55 PDT  Opened input file "prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1638 traces, 1638 summary
Retagger: tagging trace set: gauss with 1322 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1322 traces tagged "gauss"
FrameSaver: q=163193 n=18219 tag=gauss
wclsFrameSaver: saving 1638 traces tagged "wiener"
FrameSaver: q=177863 n=16197 tag=wiener
28-Mar-2025 09:56:50 PDT  Opened output file with pattern "prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1672 traces, 1672 summary
Retagger: tagging trace set: gauss with 1352 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1352 traces tagged "gauss"
FrameSaver: q=159645 n=17705 tag=gauss
wclsFrameSaver: saving 1672 traces tagged "wiener"
FrameSaver: q=175821 n=15701 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1762 traces, 1762 summary
Retagger: tagging trace set: gauss with 1390 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1390 traces tagged "gauss"
FrameSaver: q=170205 n=18812 tag=gauss
wclsFrameSaver: saving 1762 traces tagged "wiener"
FrameSaver: q=187878 n=16673 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1764 traces, 1764 summary
Retagger: tagging trace set: gauss with 1323 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1323 traces tagged "gauss"
FrameSaver: q=160108 n=18328 tag=gauss
wclsFrameSaver: saving 1764 traces tagged "wiener"
FrameSaver: q=177991 n=15998 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1651 traces, 1651 summary
Retagger: tagging trace set: gauss with 1405 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1405 traces tagged "gauss"
FrameSaver: q=159748 n=18291 tag=gauss
wclsFrameSaver: saving 1651 traces tagged "wiener"
FrameSaver: q=174662 n=16454 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1810 traces, 1810 summary
Retagger: tagging trace set: gauss with 1360 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1360 traces tagged "gauss"
FrameSaver: q=163004 n=18796 tag=gauss
wclsFrameSaver: saving 1810 traces tagged "wiener"
FrameSaver: q=181509 n=16335 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1700 traces, 1700 summary
Retagger: tagging trace set: gauss with 1310 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1310 traces tagged "gauss"
FrameSaver: q=163377 n=18284 tag=gauss
wclsFrameSaver: saving 1700 traces tagged "wiener"
FrameSaver: q=179958 n=16008 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1664 traces, 1664 summary
Retagger: tagging trace set: gauss with 1309 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1309 traces tagged "gauss"
FrameSaver: q=158718 n=17590 tag=gauss
wclsFrameSaver: saving 1664 traces tagged "wiener"
FrameSaver: q=174670 n=15489 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1669 traces, 1669 summary
Retagger: tagging trace set: gauss with 1371 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1371 traces tagged "gauss"
FrameSaver: q=158433 n=17845 tag=gauss
wclsFrameSaver: saving 1669 traces tagged "wiener"
FrameSaver: q=174140 n=15781 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1689 traces, 1689 summary
Retagger: tagging trace set: gauss with 1274 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1274 traces tagged "gauss"
FrameSaver: q=162511 n=18205 tag=gauss
wclsFrameSaver: saving 1689 traces tagged "wiener"
FrameSaver: q=178226 n=15877 tag=wiener
28-Mar-2025 10:05:17 PDT  Closed output file "prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim_reco.root"
28-Mar-2025 10:05:17 PDT  Closed input file "prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim.root"

=================================================================================================================================
TimeTracker printout (sec)                         Min           Avg           Max         Median          RMS         nEvts   
=================================================================================================================================
Full event                                       54.3554       55.9286       57.6119       55.9788       1.24065        10     
---------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.000399612   0.000551906   0.000824054   0.000528737   0.000123552      10     
reco:rns:RandomNumberSaver                     3.6104e-05    6.54956e-05   0.00027998    4.0359e-05    7.16528e-05      10     
reco:wclsdatavd:WireCellToolkit                  53.1502       55.2849       57.1419       55.3163       1.39973        10     
reco:gaushit:GausHitFinder                      0.0200771     0.0215848     0.0247469     0.0210752    0.00138967       10     
reco:reco3d:SpacePointSolver                    0.0735916     0.0820317     0.0966678     0.0814841    0.00746891       10     
reco:hitpdune:DisambigFromSpacePoints          0.00202744    0.00265917    0.00419802    0.00244241    0.000573337      10     
reco:pandora:StandardPandora                    0.287853      0.487478       1.94499       0.32961      0.486189        10     
reco:pandoraTrack:LArPandoraTrackCreation      0.00168442    0.00283279    0.00515184    0.00288808    0.000903099      10     
reco:pandoraShower:LArPandoraShowerCreation    0.000273296   0.000329425   0.000743266   0.000284745   0.000138171      10     
reco:pandoraStdcalo:Calorimetry                0.00178206    0.00222838    0.00394659    0.00202501    0.00059378       10     
reco:pandoraGnocalo:GnocchiCalorimetry         0.00120707    0.00142607    0.00290446    0.00126804    0.000494517      10     
[art]:TriggerResults:TriggerResultInserter     1.7454e-05    2.25866e-05   5.3516e-05    1.8313e-05    1.04603e-05      10     
end_path:out1:RootOutput                        3.562e-06     8.692e-06    1.9722e-05     8.038e-06    3.91928e-06      10     
end_path:out1:RootOutput(write)                 0.0375743     0.0388422     0.0403062     0.0388289    0.000876495      10     
=================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3307.08 MB
  Peak resident set size usage (VmHWM): 1552.4 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_muon_m_450MeV_protodunevd_1_167106_42_1743180364_gen_g4_stage1_g4_stage2_detsim_reco.root', 'start_time': 1743180916.439911, 'end_time': 1743181518.0781024}
justIN time: 2025-04-03 07:45:50 UTC       justIN version: 01.03.00