justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 166665.1@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID166665.1@justin-prod-sched02.dune.hep.ac.uk
Workflow ID5747
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-03-27 11:53:36
SiteUS_PuertoRico
EntryCMSHTPC_T3_US_PuertoRico_UPRM
Last heartbeat2025-03-27 13:07:35
From worker nodeHostnamecompute-1-3.local
cpuinfoIntel(R) Xeon(R) CPU W3550 @ 3.07GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6815744000 (6500 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-03-27 12:38:05
Input filesmonte-carlo-005747-000396
JobscriptExit code0
Real time28m (1689s)
CPU time27m (1673s = 99%)
Max RSS bytes1622781952 (1547 MiB)
Outputting started2025-03-27 13:06:15
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim_reco.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim_reco.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim_reco.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05747/1/007/prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim.root
Finished2025-03-27 13:07:35
Saved logsjustin-logs:166665.1-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

02279e+07
Now with regularization...
Begin: 3.5581e+07
0 3.5575e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
Art has completed and will exit with status 0.

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  pandoraWriter
  pandoracalipid
  pandorapid
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/8e14053062785f80c03da35c9e53597243e9b1b5/protodunevd_v4_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16499 nodes/ 2408 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
27-Mar-2025 08:52:57 AST  Initiating request to open input file "prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim.root"
27-Mar-2025 08:52:57 AST  Opened input file "prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3970 traces, 3970 summary
Retagger: tagging trace set: gauss with 2495 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2495 traces tagged "gauss"
FrameSaver: q=429365 n=38656 tag=gauss
wclsFrameSaver: saving 3970 traces tagged "wiener"
FrameSaver: q=469056 n=31867 tag=wiener
27-Mar-2025 08:54:12 AST  Opened output file with pattern "prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 4031 traces, 4031 summary
Retagger: tagging trace set: gauss with 2532 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2532 traces tagged "gauss"
FrameSaver: q=436118 n=38784 tag=gauss
wclsFrameSaver: saving 4031 traces tagged "wiener"
FrameSaver: q=478523 n=31905 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 5141 traces, 5141 summary
Retagger: tagging trace set: gauss with 3190 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 3190 traces tagged "gauss"
FrameSaver: q=711501 n=55134 tag=gauss
wclsFrameSaver: saving 5141 traces tagged "wiener"
FrameSaver: q=760798 n=46113 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3939 traces, 3939 summary
Retagger: tagging trace set: gauss with 2472 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2472 traces tagged "gauss"
FrameSaver: q=423287 n=37223 tag=gauss
wclsFrameSaver: saving 3939 traces tagged "wiener"
FrameSaver: q=464094 n=30633 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 4561 traces, 4561 summary
Retagger: tagging trace set: gauss with 2888 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2888 traces tagged "gauss"
FrameSaver: q=609264 n=48108 tag=gauss
wclsFrameSaver: saving 4561 traces tagged "wiener"
FrameSaver: q=656633 n=40125 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3979 traces, 3979 summary
Retagger: tagging trace set: gauss with 2452 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2452 traces tagged "gauss"
FrameSaver: q=418603 n=38270 tag=gauss
wclsFrameSaver: saving 3979 traces tagged "wiener"
FrameSaver: q=459655 n=31290 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 4679 traces, 4679 summary
Retagger: tagging trace set: gauss with 2857 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2857 traces tagged "gauss"
FrameSaver: q=610552 n=49388 tag=gauss
wclsFrameSaver: saving 4679 traces tagged "wiener"
FrameSaver: q=656197 n=41037 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3945 traces, 3945 summary
Retagger: tagging trace set: gauss with 2458 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2458 traces tagged "gauss"
FrameSaver: q=425251 n=37903 tag=gauss
wclsFrameSaver: saving 3945 traces tagged "wiener"
FrameSaver: q=465285 n=31024 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3912 traces, 3912 summary
Retagger: tagging trace set: gauss with 2413 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2413 traces tagged "gauss"
FrameSaver: q=426612 n=37359 tag=gauss
wclsFrameSaver: saving 3912 traces tagged "wiener"
FrameSaver: q=466760 n=30630 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 4181 traces, 4181 summary
Retagger: tagging trace set: gauss with 2611 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2611 traces tagged "gauss"
FrameSaver: q=490143 n=40810 tag=gauss
wclsFrameSaver: saving 4181 traces tagged "wiener"
FrameSaver: q=532781 n=33646 tag=wiener
27-Mar-2025 09:06:13 AST  Closed output file "prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim_reco.root"
27-Mar-2025 09:06:13 AST  Closed input file "prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim.root"

=================================================================================================================================
TimeTracker printout (sec)                         Min           Avg           Max         Median          RMS         nEvts   
=================================================================================================================================
Full event                                       69.1837       78.824        89.5752        76.2         6.73913        10     
---------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.000393166   0.000668724   0.00107483    0.000625509   0.000216305      10     
reco:rns:RandomNumberSaver                     3.8992e-05    6.33811e-05   0.000249245   3.96155e-05   6.22316e-05      10     
reco:wclsdatavd:WireCellToolkit                  68.5108       77.8482       88.2275       75.327        6.72226        10     
reco:gaushit:GausHitFinder                      0.0405793     0.0583661     0.120604      0.0451353     0.0241146       10     
reco:reco3d:SpacePointSolver                    0.271702      0.322506      0.464293      0.282492      0.0646958       10     
reco:hitpdune:DisambigFromSpacePoints          0.00491671    0.00675126     0.0106377    0.00593694    0.00193451       10     
reco:pandora:StandardPandora                    0.233188       0.47473       1.77545      0.305184      0.444975        10     
reco:pandoraTrack:LArPandoraTrackCreation      0.00480913     0.0101424     0.0257203    0.00827337    0.00583941       10     
reco:pandoraShower:LArPandoraShowerCreation     0.0022423    0.00576383     0.0173317    0.00387458    0.00463916       10     
reco:pandoraStdcalo:Calorimetry                0.00891188     0.0131935     0.033591      0.0105676    0.00711964       10     
reco:pandoraGnocalo:GnocchiCalorimetry         0.00302599    0.00448806     0.0104225    0.00340905    0.00225996       10     
[art]:TriggerResults:TriggerResultInserter     1.8805e-05    3.04426e-05   5.8546e-05    2.5949e-05    1.29066e-05      10     
end_path:out1:RootOutput                        3.906e-06    5.8461e-06    1.7061e-05     4.543e-06    3.77472e-06      10     
end_path:out1:RootOutput(write)                 0.056046      0.0732068     0.121653      0.0606819     0.0234041       10     
=================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3402.6 MB
  Peak resident set size usage (VmHWM): 1423.1 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_muon_5GeV_protodunevd_1_166665_1_1743079093_gen_g4_stage1_g4_stage2_detsim_reco.root', 'start_time': 1743079939.27237, 'end_time': 1743080774.8378513}
justIN time: 2025-04-03 08:13:14 UTC       justIN version: 01.03.00