justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 358069.74@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID358069.74@justin-prod-sched01.dune.hep.ac.uk
Workflow ID5747
Stage ID1
User nameykermaid@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes6815744000 (6500 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-03-27 11:15:55
SiteUK_RAL-PPD
EntryCMSHTPC_T2_UK_SGrid_RALPP_hep206
Last heartbeat2025-03-27 12:20:19
From worker nodeHostnameheplnc163.pp.rl.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes6815744000 (6500 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-03-27 12:00:19
Input filesmonte-carlo-005747-000229
JobscriptExit code0
Real time19m (1172s)
CPU time18m (1125s = 95%)
Max RSS bytes1712259072 (1632 MiB)
Outputting started2025-03-27 12:19:52
Output files
Finished2025-03-27 12:20:13
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ow with regularization...
Begin: 1.97286e+07
0 1.97262e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
Art has completed and will exit with status 0.

Stderr:
The following module labels are either not assigned to any path,
or they have been assigned to ignored path(s):
  emtrkmichelid
  nhitsfilter
  pandoraWriter
  pandoracalipid
  pandorapid
  trigfilter
Info in <TGeoManager::Import>: Reading geometry from file: /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/8e14053062785f80c03da35c9e53597243e9b1b5/protodunevd_v4_refactored.gdml
Info in <TGeoManager::TGeoManager>: Geometry GDMLImport, Geometry imported from GDML created
Info in <TGeoManager::SetTopVolume>: Top volume is volWorld. Master volume is volWorld
Info in <TGeoNavigator::BuildCache>: --- Maximum geometry depth set to 100
Info in <TGeoManager::CheckGeometry>: Fixing runtime shapes...
Info in <TGeoManager::CheckGeometry>: ...Nothing to fix
Info in <TGeoManager::CloseGeometry>: Counting nodes...
Info in <TGeoManager::Voxelize>: Voxelizing...
Info in <TGeoManager::CloseGeometry>: Building cache...
Info in <TGeoManager::CountLevels>: max level = 5, max placements = 292
Info in <TGeoManager::CloseGeometry>: 16499 nodes/ 2408 volume UID's in Geometry imported from GDML
Info in <TGeoManager::CloseGeometry>: ----------------modeler ready----------------
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_U (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_V (Potential memory leak).
Warning in <TFile::Append>: Replacing existing TH1: FieldResponse_Y (Potential memory leak).
Inputer: "wclsRawFrameSource"
Outputer: "wclsFrameSaver:spsaver"
wclsFrameSaver: promising to produce recob::Wires named "gauss"
wclsFrameSaver: promising to produce recob::Wires named "wiener"
27-Mar-2025 12:12:45 GMT  Initiating request to open input file "prod_muon_5GeV_protodunevd_1_358069_74_1743076829_gen_g4_stage1_g4_stage2_detsim.root"
27-Mar-2025 12:12:46 GMT  Opened input file "prod_muon_5GeV_protodunevd_1_358069_74_1743076829_gen_g4_stage1_g4_stage2_detsim.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 4226 traces, 4226 summary
Retagger: tagging trace set: gauss with 2670 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2670 traces tagged "gauss"
FrameSaver: q=524359 n=42790 tag=gauss
wclsFrameSaver: saving 4226 traces tagged "wiener"
FrameSaver: q=567818 n=35727 tag=wiener
27-Mar-2025 12:13:26 GMT  Opened output file with pattern "prod_muon_5GeV_protodunevd_1_358069_74_1743076829_gen_g4_stage1_g4_stage2_detsim_reco.root"
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 4143 traces, 4143 summary
Retagger: tagging trace set: gauss with 2605 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2605 traces tagged "gauss"
FrameSaver: q=463693 n=40867 tag=gauss
wclsFrameSaver: saving 4143 traces tagged "wiener"
FrameSaver: q=505938 n=33619 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 4026 traces, 4026 summary
Retagger: tagging trace set: gauss with 2530 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2530 traces tagged "gauss"
FrameSaver: q=441830 n=39050 tag=gauss
wclsFrameSaver: saving 4026 traces tagged "wiener"
FrameSaver: q=483617 n=32217 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3954 traces, 3954 summary
Retagger: tagging trace set: gauss with 2510 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2510 traces tagged "gauss"
FrameSaver: q=436727 n=38560 tag=gauss
wclsFrameSaver: saving 3954 traces tagged "wiener"
FrameSaver: q=477665 n=31780 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3961 traces, 3961 summary
Retagger: tagging trace set: gauss with 2494 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2494 traces tagged "gauss"
FrameSaver: q=420972 n=37985 tag=gauss
wclsFrameSaver: saving 3961 traces tagged "wiener"
FrameSaver: q=461218 n=31416 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3963 traces, 3963 summary
Retagger: tagging trace set: gauss with 2471 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2471 traces tagged "gauss"
FrameSaver: q=423544 n=37943 tag=gauss
wclsFrameSaver: saving 3963 traces tagged "wiener"
FrameSaver: q=464072 n=31088 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3865 traces, 3865 summary
Retagger: tagging trace set: gauss with 2348 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2348 traces tagged "gauss"
FrameSaver: q=418309 n=36732 tag=gauss
wclsFrameSaver: saving 3865 traces tagged "wiener"
FrameSaver: q=458398 n=30046 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 4110 traces, 4110 summary
Retagger: tagging trace set: gauss with 2510 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2510 traces tagged "gauss"
FrameSaver: q=514504 n=40948 tag=gauss
wclsFrameSaver: saving 4110 traces tagged "wiener"
FrameSaver: q=557549 n=33734 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 4010 traces, 4010 summary
Retagger: tagging trace set: gauss with 2532 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2532 traces tagged "gauss"
FrameSaver: q=433782 n=38810 tag=gauss
wclsFrameSaver: saving 4010 traces tagged "wiener"
FrameSaver: q=475101 n=32159 tag=wiener
RawFrameSource: got 12288 raw::RawDigit objects
	input nticks=6000 keeping as is
Retagger: tagging trace set: wiener with 3954 traces, 3954 summary
Retagger: tagging trace set: gauss with 2465 traces, 0 summary
wclsFrameSaver saving cooked to 6000 ticks
wclsFrameSaver: saving 2465 traces tagged "gauss"
FrameSaver: q=429309 n=38316 tag=gauss
wclsFrameSaver: saving 3954 traces tagged "wiener"
FrameSaver: q=470022 n=31455 tag=wiener
27-Mar-2025 12:19:51 GMT  Closed output file "prod_muon_5GeV_protodunevd_1_358069_74_1743076829_gen_g4_stage1_g4_stage2_detsim_reco.root"
27-Mar-2025 12:19:51 GMT  Closed input file "prod_muon_5GeV_protodunevd_1_358069_74_1743076829_gen_g4_stage1_g4_stage2_detsim.root"

=================================================================================================================================
TimeTracker printout (sec)                         Min           Avg           Max         Median          RMS         nEvts   
=================================================================================================================================
Full event                                       40.0879       41.7254       43.7557       41.5815       1.00906        10     
---------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.00045773    0.000670837   0.000947932   0.000677043   0.000144493      10     
reco:rns:RandomNumberSaver                     4.9062e-05    9.34725e-05   0.000430799    5.866e-05    0.000112554      10     
reco:wclsdatavd:WireCellToolkit                  37.4005       40.8384       43.0351       40.9428       1.41836        10     
reco:gaushit:GausHitFinder                      0.0405917     0.0483544     0.0639896     0.046861     0.00696239       10     
reco:reco3d:SpacePointSolver                     0.22751      0.323861      0.364257      0.332205      0.0379692       10     
reco:hitpdune:DisambigFromSpacePoints          0.00297957     0.0053735    0.00912383    0.00520458    0.00142686       10     
reco:pandora:StandardPandora                    0.128468      0.412054       2.14129      0.223233      0.577591        10     
reco:pandoraTrack:LArPandoraTrackCreation       0.0039237    0.00606922    0.00992473    0.00600893    0.00157522       10     
reco:pandoraShower:LArPandoraShowerCreation    0.00132883    0.00333742    0.00739349    0.00305337    0.00172921       10     
reco:pandoraStdcalo:Calorimetry                0.00469648    0.00832732     0.0121195     0.0083875    0.00171862       10     
reco:pandoraGnocalo:GnocchiCalorimetry         0.00205481    0.00409981    0.00696616    0.00396707    0.00112528       10     
[art]:TriggerResults:TriggerResultInserter     1.2794e-05    2.66451e-05   7.5542e-05    2.1245e-05    1.68937e-05      10     
end_path:out1:RootOutput                        8.035e-06    1.5966e-05    5.6717e-05     9.623e-06    1.46988e-05      10     
end_path:out1:RootOutput(write)                 0.050503       0.06964      0.0996605     0.0676722     0.0148644       10     
=================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 3435.59 MB
  Peak resident set size usage (VmHWM): 1618.77 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'prod_muon_5GeV_protodunevd_1_358069_74_1743076829_gen_g4_stage1_g4_stage2_detsim_reco.root', 'start_time': 1743077529.159159, 'end_time': 1743077992.0743306}
justIN time: 2025-04-03 08:25:27 UTC       justIN version: 01.03.00