justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 167101.22@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID167101.22@justin-prod-sched02.dune.hep.ac.uk
Workflow ID5781
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-03-28 15:49:00
SiteUS_PuertoRico
EntryCMSHTPC_T3_US_PuertoRico_UPRM
Last heartbeat2025-03-28 16:58:13
From worker nodeHostnamecompute-1-3.local
cpuinfoIntel(R) Xeon(R) CPU W3550 @ 3.07GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6815744000 (6500 MiB)
Wall seconds limit171000 (47 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-03-28 16:31:39
Input filesmonte-carlo-005781-000484
JobscriptExit code0
Real time25m (1527s)
CPU time25m (1500s = 98%)
Max RSS bytes1599553536 (1525 MiB)
Outputting started2025-03-28 16:57:07
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim_reco.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim_reco.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim_reco.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen.err
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1.log
https://fndcadoor.fnal.gov:2880/dune/scratch/users/ykermaid/pdvd_muon/05781/1/011/prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim.err
Finished2025-03-28 16:58:13
Saved logsjustin-logs:167101.22-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

tion...
Begin: 2.05355e+07
0 2.01235e+07
1 2.01215e+07
Now with regularization...
Begin: 1.93613e+07
0 1.93613e+07
Art has completed and will exit with status 0.

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  pandoraWriter
  pandoracalipid
  pandorapid
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser2.opensciencegrid.org/sw/dune/497fe0b9af7ebfe5ac11fff59041e9416f4d09cb/protodunevd_v4_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16499 nodes/ 2408 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
28-Mar-2025 12:44:09 AST  Initiating request to open input file "prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim.root"
28-Mar-2025 12:44:09 AST  Opened input file "prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1773 traces, 1773 summary
Retagger: tagging trace set: gauss with 1410 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1410 traces tagged "gauss"
FrameSaver: q=168467 n=19015 tag=gauss
wclsFrameSaver: saving 1773 traces tagged "wiener"
FrameSaver: q=184800 n=16860 tag=wiener
28-Mar-2025 12:45:34 AST  Opened output file with pattern "prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1696 traces, 1696 summary
Retagger: tagging trace set: gauss with 1405 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1405 traces tagged "gauss"
FrameSaver: q=171039 n=19152 tag=gauss
wclsFrameSaver: saving 1696 traces tagged "wiener"
FrameSaver: q=185823 n=17105 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1854 traces, 1854 summary
Retagger: tagging trace set: gauss with 1465 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1465 traces tagged "gauss"
FrameSaver: q=174517 n=19902 tag=gauss
wclsFrameSaver: saving 1854 traces tagged "wiener"
FrameSaver: q=191663 n=17746 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1789 traces, 1789 summary
Retagger: tagging trace set: gauss with 1349 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1349 traces tagged "gauss"
FrameSaver: q=171961 n=18700 tag=gauss
wclsFrameSaver: saving 1789 traces tagged "wiener"
FrameSaver: q=190303 n=16417 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1724 traces, 1724 summary
Retagger: tagging trace set: gauss with 1390 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1390 traces tagged "gauss"
FrameSaver: q=164752 n=18737 tag=gauss
wclsFrameSaver: saving 1724 traces tagged "wiener"
FrameSaver: q=180322 n=16699 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1715 traces, 1715 summary
Retagger: tagging trace set: gauss with 1422 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1422 traces tagged "gauss"
FrameSaver: q=176232 n=18733 tag=gauss
wclsFrameSaver: saving 1715 traces tagged "wiener"
FrameSaver: q=191868 n=16602 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1790 traces, 1790 summary
Retagger: tagging trace set: gauss with 1394 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1394 traces tagged "gauss"
FrameSaver: q=172226 n=18980 tag=gauss
wclsFrameSaver: saving 1790 traces tagged "wiener"
FrameSaver: q=190104 n=16719 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1777 traces, 1777 summary
Retagger: tagging trace set: gauss with 1440 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1440 traces tagged "gauss"
FrameSaver: q=172374 n=19290 tag=gauss
wclsFrameSaver: saving 1777 traces tagged "wiener"
FrameSaver: q=188621 n=17069 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1673 traces, 1673 summary
Retagger: tagging trace set: gauss with 1366 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1366 traces tagged "gauss"
FrameSaver: q=173321 n=19409 tag=gauss
wclsFrameSaver: saving 1673 traces tagged "wiener"
FrameSaver: q=186954 n=17341 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 1748 traces, 1748 summary
Retagger: tagging trace set: gauss with 1455 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 1455 traces tagged "gauss"
FrameSaver: q=173700 n=19183 tag=gauss
wclsFrameSaver: saving 1748 traces tagged "wiener"
FrameSaver: q=189114 n=17276 tag=wiener
28-Mar-2025 12:57:04 AST  Closed output file "prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim_reco.root"
28-Mar-2025 12:57:04 AST  Closed input file "prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim.root"

=================================================================================================================================
TimeTracker printout (sec)                         Min           Avg           Max         Median          RMS         nEvts   
=================================================================================================================================
Full event                                       72.7216       76.1534       83.7196       73.8448       3.84321        10     
---------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.000293524   0.00040346    0.000663859   0.000362692   0.000107153      10     
reco:rns:RandomNumberSaver                     3.4806e-05    6.00889e-05   0.000242557   4.06985e-05   6.08704e-05      10     
reco:wclsdatavd:WireCellToolkit                  72.2782       75.1847       83.3348       72.8273       3.83576        10     
reco:gaushit:GausHitFinder                      0.0210244     0.0221675     0.0242358     0.0218273    0.000910936      10     
reco:reco3d:SpacePointSolver                    0.0613828     0.0675369     0.0762872     0.0668093    0.00520541       10     
reco:hitpdune:DisambigFromSpacePoints          0.00168899     0.0022416    0.00399248    0.00205851    0.000605575      10     
reco:pandora:StandardPandora                    0.234119      0.832938       5.96003      0.262378       1.70911        10     
reco:pandoraTrack:LArPandoraTrackCreation      0.00152888    0.00274772    0.00455307     0.0028365    0.00085938       10     
reco:pandoraShower:LArPandoraShowerCreation    0.00022118    0.000391666   0.00137996    0.00023652    0.00034991       10     
reco:pandoraStdcalo:Calorimetry                0.00148188    0.00211399    0.00406125     0.001922     0.000689157      10     
reco:pandoraGnocalo:GnocchiCalorimetry         0.00101354    0.00123023    0.00249925    0.00109563    0.000426414      10     
[art]:TriggerResults:TriggerResultInserter     1.8318e-05    2.16701e-05   4.3246e-05    1.89955e-05   7.31977e-06      10     
end_path:out1:RootOutput                        3.936e-06    5.0796e-06    1.3937e-05     4.053e-06    2.95641e-06      10     
end_path:out1:RootOutput(write)                 0.0311905     0.0330676     0.0403607     0.0324193    0.00254214       10     
=================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3356.62 MB
  Peak resident set size usage (VmHWM): 1377.01 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_muon_p_450MeV_protodunevd_1_167101_22_1743179510_gen_g4_stage1_g4_stage2_detsim_reco.root', 'start_time': 1743180210.285492, 'end_time': 1743181025.924442}
justIN time: 2025-04-03 08:23:25 UTC       justIN version: 01.03.00