justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 419148.17@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID419148.17@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7814
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-20 20:19:41
SiteCZ_FZU
EntryNova_CZ_FZU_osgce1
Last heartbeat2025-06-21 05:14:58
From worker nodeHostnamemahagon07.farm.particle.cz
cpuinfoAMD EPYC 7301 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes8387559424 (7999 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-21 02:43:55
Input fileshd-protodune:np04hd_raw_run027087_0371_dataflow0_datawriter_0_20240614T180435.hdf5
JobscriptExit code0
Real time2h (9030s)
CPU time2h (8572s = 94%)
Max RSS bytes2710519808 (2584 MiB)
Outputting started2025-06-21 05:14:27
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/new_data_proc_v10_04_06d00/07814/1/002/pdhd_processing_1_419148_17_1750473849_stage1_stage2.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/new_data_proc_v10_04_06d00/07814/1/002/pdhd_processing_1_419148_17_1750473849_stage1_stage2_tfile.root
Finished2025-06-21 05:14:58
Saved logsjustin-logs:419148.17-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

q=2.63692e+06 n=1026244 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 82464 traces, 82464 summary
Retagger: tagging trace set: gauss with 67666 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 67666 traces tagged "gauss"
FrameSaver: q=2.00815e+06 n=1120765 tag=gauss
wclsFrameSaver: saving 82464 traces tagged "wiener"
FrameSaver: q=2.26234e+06 n=1069195 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 71720 traces, 71720 summary
Retagger: tagging trace set: gauss with 57193 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 57193 traces tagged "gauss"
FrameSaver: q=1.3217e+06 n=827760 tag=gauss
wclsFrameSaver: saving 71720 traces tagged "wiener"
FrameSaver: q=1.55864e+06 n=797562 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 74764 traces, 74764 summary
Retagger: tagging trace set: gauss with 58385 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58385 traces tagged "gauss"
FrameSaver: q=1.74509e+06 n=912681 tag=gauss
wclsFrameSaver: saving 74764 traces tagged "wiener"
FrameSaver: q=2.0298e+06 n=886746 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 63850 traces, 63850 summary
Retagger: tagging trace set: gauss with 51582 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51582 traces tagged "gauss"
FrameSaver: q=1.23264e+06 n=773578 tag=gauss
wclsFrameSaver: saving 63850 traces tagged "wiener"
FrameSaver: q=1.41453e+06 n=739277 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83109 traces, 83109 summary
Retagger: tagging trace set: gauss with 63323 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 63323 traces tagged "gauss"
FrameSaver: q=1.87759e+06 n=948666 tag=gauss
wclsFrameSaver: saving 83109 traces tagged "wiener"
FrameSaver: q=2.22602e+06 n=928136 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67833 traces, 67833 summary
Retagger: tagging trace set: gauss with 56200 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56200 traces tagged "gauss"
FrameSaver: q=1.43729e+06 n=845025 tag=gauss
wclsFrameSaver: saving 67833 traces tagged "wiener"
FrameSaver: q=1.67644e+06 n=816955 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 81716 traces, 81716 summary
Retagger: tagging trace set: gauss with 62949 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 62949 traces tagged "gauss"
FrameSaver: q=1.80249e+06 n=916322 tag=gauss
wclsFrameSaver: saving 81716 traces tagged "wiener"
FrameSaver: q=2.17839e+06 n=891474 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 89364 traces, 89364 summary
Retagger: tagging trace set: gauss with 68878 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 68878 traces tagged "gauss"
FrameSaver: q=2.11891e+06 n=1091589 tag=gauss
wclsFrameSaver: saving 89364 traces tagged "wiener"
FrameSaver: q=2.52148e+06 n=1061260 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 75207 traces, 75207 summary
Retagger: tagging trace set: gauss with 58640 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58640 traces tagged "gauss"
FrameSaver: q=1.69225e+06 n=902619 tag=gauss
wclsFrameSaver: saving 75207 traces tagged "wiener"
FrameSaver: q=1.9226e+06 n=867748 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 90086 traces, 90086 summary
Retagger: tagging trace set: gauss with 70240 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 70240 traces tagged "gauss"
FrameSaver: q=1.96624e+06 n=1055420 tag=gauss
wclsFrameSaver: saving 90086 traces tagged "wiener"
FrameSaver: q=2.30758e+06 n=1002924 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83221 traces, 83221 summary
Retagger: tagging trace set: gauss with 66761 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 66761 traces tagged "gauss"
FrameSaver: q=1.89706e+06 n=1051781 tag=gauss
wclsFrameSaver: saving 83221 traces tagged "wiener"
FrameSaver: q=2.15625e+06 n=995650 tag=wiener
21-Jun-2025 07:14:25 CEST  Closed output file "pdhd_processing_1_419148_17_1750473849_stage1_stage2.root"
21-Jun-2025 07:14:25 CEST  Closed input file "pdhd_processing_1_419148_17_1750473849_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              128.175       186.93        493.607       171.315       58.6368        40     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000645925   0.00321838     0.0115377    0.00274605    0.00201992       40     
produce:fembfilter:PDHDFEMBFilter                       0.77327      0.993127       1.29903       1.05738      0.144179        40     
produce:wclsdatahd:WireCellToolkit                      39.2221       51.8118       63.4382       51.6941       6.52304        40     
produce:gaushit:GausHitFinder                           0.53262       1.05946       1.5889        1.02244      0.259557        40     
produce:nhitsfilter:NumberOfHitsFilter                0.000294935   0.000422567   0.000757245   0.00043043    8.33184e-05      40     
produce:reco3d:SpacePointSolver                        0.330103       1.0908        5.05809      0.737999      0.954668        40     
produce:hitpdune:DisambigFromSpacePoints               0.179748      0.700108       1.44927      0.622259      0.329314        40     
produce:pandora:StandardPandora                         12.6932       32.2095       298.106       23.4511       43.5273        40     
produce:pandoraWriter:StandardPandora                  0.135105      0.233047      0.346433      0.222446      0.055278        40     
produce:pandoraTrack:LArPandoraTrackCreation           0.776716       2.25926       3.9954        2.06923      0.780417        40     
produce:pandoraShower:LArPandoraShowerCreation          0.55624       2.76377       6.05572       2.67791       1.27988        40     
produce:pandoracalo:Calorimetry                        0.314138       1.0582        1.93522      0.982955       0.38069        40     
produce:pandoracalonosce:Calorimetry                   0.292929      0.966385       1.84783      0.906719      0.364122        40     
produce:pandorapid:Chi2ParticleID                     0.00129766    0.00259505    0.00436292    0.00235213    0.000667722      40     
produce:pandoraShowercalo:ShowerCalorimetry             0.56494       2.48764       5.57216       2.25148       1.18299        40     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.532752       2.33878       5.59612       2.15753       1.1083         40     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0683748     0.185896      0.300462      0.185016      0.0494472       40     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0237125     0.0662251     0.109251      0.0671356     0.0192023       40     
produce:emtrkmichelid:EmTrackMichelId                   44.2552       65.5425       95.7159       65.1229       12.7542        40     
[art]:TriggerResults:TriggerResultInserter            2.4235e-05    4.64325e-05   0.000105198   4.04505e-05   1.95926e-05      40     
end_path:out1:RootOutput                               8.356e-06    1.56821e-05   5.4082e-05    1.3034e-05    8.18648e-06      40     
end_path:michelremoving:michelremoving                 0.0281233      0.12259      0.479292      0.0598469     0.107763        40     
end_path:michelremovinggnocchi:michelremoving          0.0248154     0.116374      0.491696      0.0537818     0.108705        40     
end_path:calibanagnocchi:CalibAnaTree                   8.50381       19.4318       48.7195       14.8815       9.46622        40     
end_path:out1:RootOutput(write)                        0.765427       1.32466       1.96507       1.30658      0.283955        40     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5108.29 MB
  Peak resident set size usage (VmHWM): 2710.52 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'pdhd_processing_1_419148_17_1750473849_stage1_stage2.root', 'tfile_out': 'pdhd_processing_1_419148_17_1750473849_stage1_stage2_tfile.root', 'start_time': 1750475333.1830986, 'end_time': 1750482866.8609254}
justIN time: 2025-08-22 03:02:44 UTC       justIN version: 01.03.02