justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 167105.160@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID167105.160@justin-prod-sched02.dune.hep.ac.uk
Workflow ID5781
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-03-28 15:51:17
SiteCA_SFU
EntryDUNE_CA_SFU_lcg-ce3
Last heartbeat2025-03-28 17:03:44
From worker nodeHostnamecdr929.int.cedar.computecanada.ca
cpuinfoIntel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6815744000 (6500 MiB)
Wall seconds limit84598 (23 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-03-28 16:43:29
Input filesmonte-carlo-005781-000879
JobscriptExit code0
Real time19m (1154s)
CPU time19m (1142s = 98%)
Max RSS bytes1729425408 (1649 MiB)
Outputting started2025-03-28 17:02:44
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim_reco.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim_reco.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/013/prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim_reco.err
Finished2025-03-28 17:03:44
Saved logsjustin-logs:167105.160-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Begin: 6.4345e+07
0 6.36233e+07
1 6.36233e+07
Now with regularization...
Begin: 6.30854e+07
0 6.30854e+07
Art has completed and will exit with status 0.

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  pandoraWriter
  pandoracalipid
  pandorapid
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/497fe0b9af7ebfe5ac11fff59041e9416f4d09cb/protodunevd_v4_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16499 nodes/ 2408 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
28-Mar-2025 09:53:13 PDT  Initiating request to open input file "prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim.root"
28-Mar-2025 09:53:13 PDT  Opened input file "prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1876 traces, 1876 summary
Retagger: tagging trace set: gauss with 1398 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1398 traces tagged "gauss"
FrameSaver: q=173598 n=19766 tag=gauss
wclsFrameSaver: saving 1876 traces tagged "wiener"
FrameSaver: q=192058 n=17268 tag=wiener
28-Mar-2025 09:54:09 PDT  Opened output file with pattern "prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1748 traces, 1748 summary
Retagger: tagging trace set: gauss with 1438 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1438 traces tagged "gauss"
FrameSaver: q=170372 n=19208 tag=gauss
wclsFrameSaver: saving 1748 traces tagged "wiener"
FrameSaver: q=185929 n=17240 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1707 traces, 1707 summary
Retagger: tagging trace set: gauss with 1427 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1427 traces tagged "gauss"
FrameSaver: q=176452 n=20167 tag=gauss
wclsFrameSaver: saving 1707 traces tagged "wiener"
FrameSaver: q=190244 n=18314 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1795 traces, 1795 summary
Retagger: tagging trace set: gauss with 1436 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1436 traces tagged "gauss"
FrameSaver: q=173341 n=19450 tag=gauss
wclsFrameSaver: saving 1795 traces tagged "wiener"
FrameSaver: q=189559 n=17222 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1697 traces, 1697 summary
Retagger: tagging trace set: gauss with 1344 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1344 traces tagged "gauss"
FrameSaver: q=170302 n=19037 tag=gauss
wclsFrameSaver: saving 1697 traces tagged "wiener"
FrameSaver: q=186297 n=16860 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1677 traces, 1677 summary
Retagger: tagging trace set: gauss with 1331 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1331 traces tagged "gauss"
FrameSaver: q=167878 n=18671 tag=gauss
wclsFrameSaver: saving 1677 traces tagged "wiener"
FrameSaver: q=183149 n=16467 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1709 traces, 1709 summary
Retagger: tagging trace set: gauss with 1422 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1422 traces tagged "gauss"
FrameSaver: q=178860 n=19911 tag=gauss
wclsFrameSaver: saving 1709 traces tagged "wiener"
FrameSaver: q=193497 n=18041 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1730 traces, 1730 summary
Retagger: tagging trace set: gauss with 1372 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1372 traces tagged "gauss"
FrameSaver: q=172746 n=18187 tag=gauss
wclsFrameSaver: saving 1730 traces tagged "wiener"
FrameSaver: q=189877 n=16107 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1824 traces, 1824 summary
Retagger: tagging trace set: gauss with 1353 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1353 traces tagged "gauss"
FrameSaver: q=166884 n=18970 tag=gauss
wclsFrameSaver: saving 1824 traces tagged "wiener"
FrameSaver: q=185398 n=16582 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1780 traces, 1780 summary
Retagger: tagging trace set: gauss with 1381 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1381 traces tagged "gauss"
FrameSaver: q=177136 n=19707 tag=gauss
wclsFrameSaver: saving 1780 traces tagged "wiener"
FrameSaver: q=193296 n=17321 tag=wiener
28-Mar-2025 10:02:43 PDT  Closed output file "prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim_reco.root"
28-Mar-2025 10:02:43 PDT  Closed input file "prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim.root"

=================================================================================================================================
TimeTracker printout (sec)                         Min           Avg           Max         Median          RMS         nEvts   
=================================================================================================================================
Full event                                       55.0426       56.7203       58.0882       56.547       0.845337        10     
---------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.000395891   0.000519696   0.000630937   0.000511865   8.15612e-05      10     
reco:rns:RandomNumberSaver                     4.1872e-05    6.2328e-05    0.000226399   4.4264e-05    5.47046e-05      10     
reco:wclsdatavd:WireCellToolkit                  52.8656       56.0552       57.6739       56.0182       1.24893        10     
reco:gaushit:GausHitFinder                      0.0221673     0.0256574     0.0290851     0.0257846    0.00234829       10     
reco:reco3d:SpacePointSolver                    0.0742892     0.0883646     0.110353      0.0853583     0.0118808       10     
reco:hitpdune:DisambigFromSpacePoints          0.00226725    0.00276165    0.00482856    0.00254285    0.000714688      10     
reco:pandora:StandardPandora                    0.242592      0.496618       1.97566      0.345148       0.49431        10     
reco:pandoraTrack:LArPandoraTrackCreation      0.00224089    0.00355735    0.00560076    0.00342083    0.000804994      10     
reco:pandoraShower:LArPandoraShowerCreation    0.000280889   0.000541352   0.00203328    0.000295667   0.000516807      10     
reco:pandoraStdcalo:Calorimetry                0.00165956    0.00243409    0.00505549    0.00214871    0.000909358      10     
reco:pandoraGnocalo:GnocchiCalorimetry          0.0011822     0.0014697    0.00305349     0.0012899    0.000531152      10     
[art]:TriggerResults:TriggerResultInserter     1.7628e-05    2.26168e-05   5.2723e-05    1.91675e-05   1.00956e-05      10     
end_path:out1:RootOutput                        7.432e-06    9.2644e-06    2.0927e-05     8.036e-06    3.89869e-06      10     
end_path:out1:RootOutput(write)                 0.037241      0.038946      0.0405579     0.0390822    0.000924855      10     
=================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3329.02 MB
  Peak resident set size usage (VmHWM): 1570.58 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_muon_p_450MeV_protodunevd_1_167105_160_1743180216_gen_g4_stage1_g4_stage2_detsim_reco.root', 'start_time': 1743180755.4642518, 'end_time': 1743181364.3633626}
justIN time: 2025-04-03 07:59:57 UTC       justIN version: 01.03.00