justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 167096.92@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID167096.92@justin-prod-sched02.dune.hep.ac.uk
Workflow ID5781
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-03-28 15:45:31
SiteUS_PuertoRico
EntryCMSHTPC_T3_US_PuertoRico_UPRM
Last heartbeat2025-03-28 16:30:25
From worker nodeHostnamecompute-1-3.local
cpuinfoIntel(R) Xeon(R) CPU W3550 @ 3.07GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6815744000 (6500 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-03-28 16:03:45
Input filesmonte-carlo-005781-000084
JobscriptExit code0
Real time25m (1535s)
CPU time25m (1506s = 98%)
Max RSS bytes1598840832 (1524 MiB)
Outputting started2025-03-28 16:29:22
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim_reco.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim_reco.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim_reco.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/003/prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1.err
Finished2025-03-28 16:30:25
Saved logsjustin-logs:167096.92-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

tion...
Begin: 1.395e+07
0 1.37905e+07
1 1.37859e+07
Now with regularization...
Begin: 1.31206e+07
0 1.31202e+07
Art has completed and will exit with status 0.

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  pandoraWriter
  pandoracalipid
  pandorapid
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/497fe0b9af7ebfe5ac11fff59041e9416f4d09cb/protodunevd_v4_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16499 nodes/ 2408 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
28-Mar-2025 12:16:23 AST  Initiating request to open input file "prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim.root"
28-Mar-2025 12:16:24 AST  Opened input file "prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1768 traces, 1768 summary
Retagger: tagging trace set: gauss with 1486 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1486 traces tagged "gauss"
FrameSaver: q=176528 n=19942 tag=gauss
wclsFrameSaver: saving 1768 traces tagged "wiener"
FrameSaver: q=191067 n=18206 tag=wiener
28-Mar-2025 12:17:41 AST  Opened output file with pattern "prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1821 traces, 1821 summary
Retagger: tagging trace set: gauss with 1470 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1470 traces tagged "gauss"
FrameSaver: q=170967 n=19240 tag=gauss
wclsFrameSaver: saving 1821 traces tagged "wiener"
FrameSaver: q=188624 n=17040 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1883 traces, 1883 summary
Retagger: tagging trace set: gauss with 1493 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1493 traces tagged "gauss"
FrameSaver: q=176155 n=20249 tag=gauss
wclsFrameSaver: saving 1883 traces tagged "wiener"
FrameSaver: q=194575 n=17838 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1804 traces, 1804 summary
Retagger: tagging trace set: gauss with 1382 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1382 traces tagged "gauss"
FrameSaver: q=177675 n=19857 tag=gauss
wclsFrameSaver: saving 1804 traces tagged "wiener"
FrameSaver: q=194355 n=17372 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1751 traces, 1751 summary
Retagger: tagging trace set: gauss with 1291 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1291 traces tagged "gauss"
FrameSaver: q=174868 n=18208 tag=gauss
wclsFrameSaver: saving 1751 traces tagged "wiener"
FrameSaver: q=191701 n=15763 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1754 traces, 1754 summary
Retagger: tagging trace set: gauss with 1397 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1397 traces tagged "gauss"
FrameSaver: q=170868 n=19040 tag=gauss
wclsFrameSaver: saving 1754 traces tagged "wiener"
FrameSaver: q=187396 n=16804 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1783 traces, 1783 summary
Retagger: tagging trace set: gauss with 1349 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1349 traces tagged "gauss"
FrameSaver: q=161915 n=18482 tag=gauss
wclsFrameSaver: saving 1783 traces tagged "wiener"
FrameSaver: q=179998 n=16217 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1863 traces, 1863 summary
Retagger: tagging trace set: gauss with 1426 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1426 traces tagged "gauss"
FrameSaver: q=169155 n=19227 tag=gauss
wclsFrameSaver: saving 1863 traces tagged "wiener"
FrameSaver: q=187278 n=16817 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1739 traces, 1739 summary
Retagger: tagging trace set: gauss with 1386 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1386 traces tagged "gauss"
FrameSaver: q=173553 n=18796 tag=gauss
wclsFrameSaver: saving 1739 traces tagged "wiener"
FrameSaver: q=190733 n=16517 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1718 traces, 1718 summary
Retagger: tagging trace set: gauss with 1394 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1394 traces tagged "gauss"
FrameSaver: q=171625 n=18936 tag=gauss
wclsFrameSaver: saving 1718 traces tagged "wiener"
FrameSaver: q=187730 n=16917 tag=wiener
28-Mar-2025 12:29:20 AST  Closed output file "prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim_reco.root"
28-Mar-2025 12:29:20 AST  Closed input file "prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim.root"

=================================================================================================================================
TimeTracker printout (sec)                         Min           Avg           Max         Median          RMS         nEvts   
=================================================================================================================================
Full event                                       71.3374       76.7956       92.4384       73.8513       6.18038        10     
---------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.000494589   0.000591561   0.000701446   0.00057958    6.47913e-05      10     
reco:rns:RandomNumberSaver                     3.4344e-05    5.99722e-05   0.000226371   3.8977e-05    5.58446e-05      10     
reco:wclsdatavd:WireCellToolkit                  70.9341       76.1154       91.9203       73.3043       6.21515        10     
reco:gaushit:GausHitFinder                      0.0205257     0.0236099     0.0347167     0.0221285    0.00394288       10     
reco:reco3d:SpacePointSolver                    0.0634646     0.0759967     0.103227      0.075162      0.0109595       10     
reco:hitpdune:DisambigFromSpacePoints           0.0020545    0.00246748    0.00413896    0.00224378    0.000602172      10     
reco:pandora:StandardPandora                    0.262465      0.532462       2.81887      0.273786       0.7624         10     
reco:pandoraTrack:LArPandoraTrackCreation      0.00154174    0.00311715     0.0056854    0.00300094    0.00106403       10     
reco:pandoraShower:LArPandoraShowerCreation    0.00023095    0.00044919     0.0014359    0.000265015   0.000383456      10     
reco:pandoraStdcalo:Calorimetry                0.00155097    0.00232779    0.00489837    0.00209525    0.000880604      10     
reco:pandoraGnocalo:GnocchiCalorimetry         0.00106739    0.00137559     0.0036524     0.0011255    0.000759854      10     
[art]:TriggerResults:TriggerResultInserter     1.8291e-05    2.6131e-05    8.8118e-05    1.9336e-05    2.0672e-05       10     
end_path:out1:RootOutput                        3.996e-06    6.2323e-06    2.2518e-05    4.5175e-06    5.43326e-06      10     
end_path:out1:RootOutput(write)                 0.0315021     0.0331111     0.0373314     0.0328555    0.00159295       10     
=================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3362.56 MB
  Peak resident set size usage (VmHWM): 1384.44 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_muon_p_450MeV_protodunevd_1_167096_92_1743177836_gen_g4_stage1_g4_stage2_detsim_reco.root', 'start_time': 1743178545.6635048, 'end_time': 1743179361.0415888}
justIN time: 2025-04-03 08:23:30 UTC       justIN version: 01.03.00