justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 167100.168@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID167100.168@justin-prod-sched02.dune.hep.ac.uk
Workflow ID5781
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-03-28 15:47:51
SiteCA_SFU
EntryDUNE_CA_SFU_lcg-ce3
Last heartbeat2025-03-28 16:49:52
From worker nodeHostnamecdr1221.int.cedar.computecanada.ca
cpuinfoIntel(R) Xeon(R) Platinum 8160 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6815744000 (6500 MiB)
Wall seconds limit84598 (23 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-03-28 16:30:22
Input filesmonte-carlo-005781-000438
JobscriptExit code0
Real time18m (1116s)
CPU time18m (1106s = 99%)
Max RSS bytes1731596288 (1651 MiB)
Outputting started2025-03-28 16:48:59
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim_reco.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim_reco.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/008/prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim_reco.err
Finished2025-03-28 16:49:52
Saved logsjustin-logs:167100.168-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

e+07
0 1.75194e+07
1 1.74274e+07
2 1.74197e+07
Now with regularization...
Begin: 1.64198e+07
0 1.64195e+07
Art has completed and will exit with status 0.

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  pandoraWriter
  pandoracalipid
  pandorapid
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/497fe0b9af7ebfe5ac11fff59041e9416f4d09cb/protodunevd_v4_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16499 nodes/ 2408 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
28-Mar-2025 09:39:59 PDT  Initiating request to open input file "prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim.root"
28-Mar-2025 09:39:59 PDT  Opened input file "prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1726 traces, 1726 summary
Retagger: tagging trace set: gauss with 1434 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1434 traces tagged "gauss"
FrameSaver: q=168690 n=19668 tag=gauss
wclsFrameSaver: saving 1726 traces tagged "wiener"
FrameSaver: q=184110 n=17741 tag=wiener
28-Mar-2025 09:40:49 PDT  Opened output file with pattern "prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1860 traces, 1860 summary
Retagger: tagging trace set: gauss with 1332 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1332 traces tagged "gauss"
FrameSaver: q=170157 n=18800 tag=gauss
wclsFrameSaver: saving 1860 traces tagged "wiener"
FrameSaver: q=189945 n=16273 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1771 traces, 1771 summary
Retagger: tagging trace set: gauss with 1452 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1452 traces tagged "gauss"
FrameSaver: q=173396 n=19511 tag=gauss
wclsFrameSaver: saving 1771 traces tagged "wiener"
FrameSaver: q=189890 n=17499 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1735 traces, 1735 summary
Retagger: tagging trace set: gauss with 1415 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1415 traces tagged "gauss"
FrameSaver: q=171632 n=19550 tag=gauss
wclsFrameSaver: saving 1735 traces tagged "wiener"
FrameSaver: q=186863 n=17400 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1877 traces, 1877 summary
Retagger: tagging trace set: gauss with 1445 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1445 traces tagged "gauss"
FrameSaver: q=174292 n=19695 tag=gauss
wclsFrameSaver: saving 1877 traces tagged "wiener"
FrameSaver: q=192515 n=17341 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1813 traces, 1813 summary
Retagger: tagging trace set: gauss with 1375 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1375 traces tagged "gauss"
FrameSaver: q=176239 n=19878 tag=gauss
wclsFrameSaver: saving 1813 traces tagged "wiener"
FrameSaver: q=192621 n=17329 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1765 traces, 1765 summary
Retagger: tagging trace set: gauss with 1306 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1306 traces tagged "gauss"
FrameSaver: q=162465 n=18200 tag=gauss
wclsFrameSaver: saving 1765 traces tagged "wiener"
FrameSaver: q=181447 n=15821 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1820 traces, 1820 summary
Retagger: tagging trace set: gauss with 1433 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1433 traces tagged "gauss"
FrameSaver: q=175158 n=19533 tag=gauss
wclsFrameSaver: saving 1820 traces tagged "wiener"
FrameSaver: q=191832 n=17121 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1801 traces, 1801 summary
Retagger: tagging trace set: gauss with 1362 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1362 traces tagged "gauss"
FrameSaver: q=169197 n=18677 tag=gauss
wclsFrameSaver: saving 1801 traces tagged "wiener"
FrameSaver: q=187391 n=16396 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1770 traces, 1770 summary
Retagger: tagging trace set: gauss with 1336 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1336 traces tagged "gauss"
FrameSaver: q=171011 n=18340 tag=gauss
wclsFrameSaver: saving 1770 traces tagged "wiener"
FrameSaver: q=189006 n=16075 tag=wiener
28-Mar-2025 09:48:57 PDT  Closed output file "prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim_reco.root"
28-Mar-2025 09:48:57 PDT  Closed input file "prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim.root"

=================================================================================================================================
TimeTracker printout (sec)                         Min           Avg           Max         Median          RMS         nEvts   
=================================================================================================================================
Full event                                       50.2359       53.5693       57.1186       53.3249       1.86159        10     
---------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.000383881   0.000535963   0.000688893   0.000537846   8.02353e-05      10     
reco:rns:RandomNumberSaver                     3.9519e-05    6.51807e-05   0.000228921   4.7306e-05    5.47577e-05      10     
reco:wclsdatavd:WireCellToolkit                  47.8472       52.8724       56.6144       52.8195       2.24505        10     
reco:gaushit:GausHitFinder                      0.0217729     0.0235682     0.0259322     0.0230622    0.00143517       10     
reco:reco3d:SpacePointSolver                    0.0789669     0.0941472     0.105065      0.0959038    0.00753971       10     
reco:hitpdune:DisambigFromSpacePoints          0.00228854    0.00271154    0.00393152    0.00257925    0.000434462      10     
reco:pandora:StandardPandora                    0.318435      0.523775       2.21306      0.337119      0.563188        10     
reco:pandoraTrack:LArPandoraTrackCreation      0.00208975     0.0033121    0.00475272    0.00340534    0.000765416      10     
reco:pandoraShower:LArPandoraShowerCreation    0.000260064   0.000439326    0.0016103    0.000288191   0.000394107      10     
reco:pandoraStdcalo:Calorimetry                0.00171385    0.00226364     0.003826     0.00219577    0.000576926      10     
reco:pandoraGnocalo:GnocchiCalorimetry         0.00112192    0.00144962    0.00281257    0.00132151    0.000458649      10     
[art]:TriggerResults:TriggerResultInserter     1.4444e-05    2.15888e-05   5.9208e-05    1.65135e-05   1.2819e-05       10     
end_path:out1:RootOutput                        7.043e-06    9.1173e-06    2.0131e-05     7.751e-06    3.75319e-06      10     
end_path:out1:RootOutput(write)                 0.0395542     0.0399747     0.0406131     0.0398386    0.00036134       10     
=================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3366.25 MB
  Peak resident set size usage (VmHWM): 1621.49 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_muon_p_450MeV_protodunevd_1_167100_168_1743179429_gen_g4_stage1_g4_stage2_detsim_reco.root', 'start_time': 1743179969.5221357, 'end_time': 1743180538.4247646}
justIN time: 2025-04-03 07:59:10 UTC       justIN version: 01.03.00