justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 167096.116@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID167096.116@justin-prod-sched02.dune.hep.ac.uk
Workflow ID5781
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-03-28 15:45:32
SiteUS_PuertoRico
EntryCMSHTPC_T3_US_PuertoRico_UPRM
Last heartbeat2025-03-28 16:30:45
From worker nodeHostnamecompute-1-3.local
cpuinfoIntel(R) Xeon(R) CPU W3550 @ 3.07GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6815744000 (6500 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-03-28 16:04:52
Input filesmonte-carlo-005781-000097
JobscriptExit code0
Real time24m (1490s)
CPU time24m (1474s = 98%)
Max RSS bytes1599643648 (1525 MiB)
Outputting started2025-03-28 16:29:44
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim_reco.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim_reco.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim_reco.root
Finished2025-03-28 16:30:45
Saved logsjustin-logs:167096.116-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Begin: 2.62698e+07
0 2.6204e+07
1 2.62037e+07
Now with regularization...
Begin: 2.55879e+07
0 2.55879e+07
Art has completed and will exit with status 0.

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  pandoraWriter
  pandoracalipid
  pandorapid
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/497fe0b9af7ebfe5ac11fff59041e9416f4d09cb/protodunevd_v4_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16499 nodes/ 2408 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
28-Mar-2025 12:17:02 AST  Initiating request to open input file "prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim.root"
28-Mar-2025 12:17:02 AST  Opened input file "prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1796 traces, 1796 summary
Retagger: tagging trace set: gauss with 1416 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1416 traces tagged "gauss"
FrameSaver: q=172018 n=19406 tag=gauss
wclsFrameSaver: saving 1796 traces tagged "wiener"
FrameSaver: q=188941 n=16929 tag=wiener
28-Mar-2025 12:18:18 AST  Opened output file with pattern "prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1729 traces, 1729 summary
Retagger: tagging trace set: gauss with 1350 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1350 traces tagged "gauss"
FrameSaver: q=171974 n=18980 tag=gauss
wclsFrameSaver: saving 1729 traces tagged "wiener"
FrameSaver: q=188108 n=16698 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1747 traces, 1747 summary
Retagger: tagging trace set: gauss with 1427 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1427 traces tagged "gauss"
FrameSaver: q=177047 n=19528 tag=gauss
wclsFrameSaver: saving 1747 traces tagged "wiener"
FrameSaver: q=192552 n=17478 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1847 traces, 1847 summary
Retagger: tagging trace set: gauss with 1364 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1364 traces tagged "gauss"
FrameSaver: q=170039 n=18995 tag=gauss
wclsFrameSaver: saving 1847 traces tagged "wiener"
FrameSaver: q=189423 n=16589 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1874 traces, 1874 summary
Retagger: tagging trace set: gauss with 1380 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1380 traces tagged "gauss"
FrameSaver: q=174850 n=19198 tag=gauss
wclsFrameSaver: saving 1874 traces tagged "wiener"
FrameSaver: q=194665 n=16768 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1739 traces, 1739 summary
Retagger: tagging trace set: gauss with 1273 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1273 traces tagged "gauss"
FrameSaver: q=175173 n=18235 tag=gauss
wclsFrameSaver: saving 1739 traces tagged "wiener"
FrameSaver: q=192815 n=15862 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1738 traces, 1738 summary
Retagger: tagging trace set: gauss with 1388 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1388 traces tagged "gauss"
FrameSaver: q=171619 n=19358 tag=gauss
wclsFrameSaver: saving 1738 traces tagged "wiener"
FrameSaver: q=187202 n=17035 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1779 traces, 1779 summary
Retagger: tagging trace set: gauss with 1406 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1406 traces tagged "gauss"
FrameSaver: q=173101 n=19756 tag=gauss
wclsFrameSaver: saving 1779 traces tagged "wiener"
FrameSaver: q=189158 n=17328 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1913 traces, 1913 summary
Retagger: tagging trace set: gauss with 1479 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1479 traces tagged "gauss"
FrameSaver: q=179854 n=19968 tag=gauss
wclsFrameSaver: saving 1913 traces tagged "wiener"
FrameSaver: q=199205 n=17572 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1774 traces, 1774 summary
Retagger: tagging trace set: gauss with 1344 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1344 traces tagged "gauss"
FrameSaver: q=173648 n=18487 tag=gauss
wclsFrameSaver: saving 1774 traces tagged "wiener"
FrameSaver: q=191000 n=16093 tag=wiener
28-Mar-2025 12:29:42 AST  Closed output file "prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim_reco.root"
28-Mar-2025 12:29:42 AST  Closed input file "prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim.root"

=================================================================================================================================
TimeTracker printout (sec)                         Min           Avg           Max         Median          RMS         nEvts   
=================================================================================================================================
Full event                                       72.9203       75.1744       87.5165       73.1406       4.30839        10     
---------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.000302014   0.000406403   0.000618754   0.00038901    8.98944e-05      10     
reco:rns:RandomNumberSaver                     3.5742e-05    5.76084e-05   0.00023686    3.73545e-05   5.97817e-05      10     
reco:wclsdatavd:WireCellToolkit                  72.4649       74.5964       87.0564       72.7332       4.3277         10     
reco:gaushit:GausHitFinder                      0.0215292     0.0240755     0.0276623     0.0236548    0.00222872       10     
reco:reco3d:SpacePointSolver                    0.0619946     0.075606      0.0916771     0.0778497    0.00887156       10     
reco:hitpdune:DisambigFromSpacePoints          0.00191846    0.00233377    0.00403638    0.00205207    0.000604824      10     
reco:pandora:StandardPandora                    0.231191      0.431007       1.85354      0.270262      0.474767        10     
reco:pandoraTrack:LArPandoraTrackCreation      0.00198675    0.00314672    0.00726825    0.00284556    0.00140965       10     
reco:pandoraShower:LArPandoraShowerCreation    0.000222251   0.00045964    0.00143217    0.00027302    0.000367856      10     
reco:pandoraStdcalo:Calorimetry                0.00162288     0.0022266    0.00444464    0.00201731    0.000759795      10     
reco:pandoraGnocalo:GnocchiCalorimetry         0.00108095    0.00132854    0.00287558    0.00117105    0.000517264      10     
[art]:TriggerResults:TriggerResultInserter     1.7183e-05    2.32299e-05   4.9647e-05     1.984e-05    9.11807e-06      10     
end_path:out1:RootOutput                        4.154e-06     5.561e-06    1.6566e-05    4.3425e-06    3.67098e-06      10     
end_path:out1:RootOutput(write)                 0.0301474     0.0330549     0.0367647     0.032862     0.00178381       10     
=================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3333.18 MB
  Peak resident set size usage (VmHWM): 1357.08 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_muon_p_450MeV_protodunevd_1_167096_116_1743177901_gen_g4_stage1_g4_stage2_detsim_reco.root', 'start_time': 1743178585.3718061, 'end_time': 1743179383.2565358}
justIN time: 2025-04-03 08:29:27 UTC       justIN version: 01.03.00