justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 418934.68@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID418934.68@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7811
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes8387559424 (7999 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-20 16:39:59
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2025-06-20 23:24:19
From worker nodeHostnamewn1208251.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes8387559424 (7999 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-20 20:13:46
Input fileshd-protodune:np04hd_raw_run027087_0126_dataflow1_datawriter_0_20240614T155303.hdf5
JobscriptExit code0
Real time3h (11375s)
CPU time2h (10752s = 94%)
Max RSS bytes2663743488 (2540 MiB)
Outputting started2025-06-20 23:23:22
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/new_data_proc_v10_04_06d00/07811/1/002/pdhd_processing_1_418934_68_1750450442_stage1_stage2_tfile.root
https://fndcadoor.fnal.gov:2880/dune/scratch/users/calcuttj/new_data_proc_v10_04_06d00/07811/1/002/pdhd_processing_1_418934_68_1750450442_stage1_stage2.root
Finished2025-06-20 23:24:19
Saved logsjustin-logs:418934.68-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

meSaver: q=1.97603e+06 n=888496 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66618 traces, 66618 summary
Retagger: tagging trace set: gauss with 53605 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 53605 traces tagged "gauss"
FrameSaver: q=1.34969e+06 n=806097 tag=gauss
wclsFrameSaver: saving 66618 traces tagged "wiener"
FrameSaver: q=1.55074e+06 n=772053 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 70354 traces, 70354 summary
Retagger: tagging trace set: gauss with 57516 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 57516 traces tagged "gauss"
FrameSaver: q=1.41927e+06 n=839541 tag=gauss
wclsFrameSaver: saving 70354 traces tagged "wiener"
FrameSaver: q=1.69612e+06 n=810308 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 80381 traces, 80381 summary
Retagger: tagging trace set: gauss with 62849 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 62849 traces tagged "gauss"
FrameSaver: q=1.68e+06 n=901653 tag=gauss
wclsFrameSaver: saving 80381 traces tagged "wiener"
FrameSaver: q=1.99664e+06 n=861394 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 82698 traces, 82698 summary
Retagger: tagging trace set: gauss with 62466 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 62466 traces tagged "gauss"
FrameSaver: q=1.87026e+06 n=906765 tag=gauss
wclsFrameSaver: saving 82698 traces tagged "wiener"
FrameSaver: q=2.23532e+06 n=889518 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 82994 traces, 82994 summary
Retagger: tagging trace set: gauss with 65182 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 65182 traces tagged "gauss"
FrameSaver: q=1.92342e+06 n=1004823 tag=gauss
wclsFrameSaver: saving 82994 traces tagged "wiener"
FrameSaver: q=2.28524e+06 n=972554 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 72733 traces, 72733 summary
Retagger: tagging trace set: gauss with 56496 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56496 traces tagged "gauss"
FrameSaver: q=1.53605e+06 n=840597 tag=gauss
wclsFrameSaver: saving 72733 traces tagged "wiener"
FrameSaver: q=1.78224e+06 n=796911 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66137 traces, 66137 summary
Retagger: tagging trace set: gauss with 55260 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 55260 traces tagged "gauss"
FrameSaver: q=1.24587e+06 n=762235 tag=gauss
wclsFrameSaver: saving 66137 traces tagged "wiener"
FrameSaver: q=1.50761e+06 n=741755 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 77485 traces, 77485 summary
Retagger: tagging trace set: gauss with 58984 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58984 traces tagged "gauss"
FrameSaver: q=1.70261e+06 n=869434 tag=gauss
wclsFrameSaver: saving 77485 traces tagged "wiener"
FrameSaver: q=2.03373e+06 n=875271 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 89543 traces, 89543 summary
Retagger: tagging trace set: gauss with 67860 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 67860 traces tagged "gauss"
FrameSaver: q=1.90677e+06 n=1007350 tag=gauss
wclsFrameSaver: saving 89543 traces tagged "wiener"
FrameSaver: q=2.26421e+06 n=967712 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 86119 traces, 86119 summary
Retagger: tagging trace set: gauss with 68417 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 68417 traces tagged "gauss"
FrameSaver: q=1.85046e+06 n=1074000 tag=gauss
wclsFrameSaver: saving 86119 traces tagged "wiener"
FrameSaver: q=2.11592e+06 n=1010885 tag=wiener
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 74454 traces, 74454 summary
Retagger: tagging trace set: gauss with 61490 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 61490 traces tagged "gauss"
FrameSaver: q=1.62542e+06 n=953093 tag=gauss
wclsFrameSaver: saving 74454 traces tagged "wiener"
FrameSaver: q=1.8668e+06 n=902031 tag=wiener
21-Jun-2025 00:23:19 BST  Closed output file "pdhd_processing_1_418934_68_1750450442_stage1_stage2.root"
21-Jun-2025 00:23:19 BST  Closed input file "pdhd_processing_1_418934_68_1750450442_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              156.265       239.808       319.644       244.203       43.8837        40     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00100105    0.00499886     0.0217332    0.00487788    0.00345799       40     
produce:fembfilter:PDHDFEMBFilter                      0.991892       1.1397        1.32864       1.13835      0.047766        40     
produce:wclsdatahd:WireCellToolkit                      62.9196       77.5852       84.9304       78.5431       5.5244         40     
produce:gaushit:GausHitFinder                          0.860477       1.3869        2.0443        1.33653      0.250246        40     
produce:nhitsfilter:NumberOfHitsFilter                0.000585164   0.000964474   0.00164952    0.000898315   0.000273455      40     
produce:reco3d:SpacePointSolver                        0.282627       1.81435       8.20531       1.37803       1.3011         40     
produce:hitpdune:DisambigFromSpacePoints               0.273798      0.955747       2.06107      0.861027      0.420332        40     
produce:pandora:StandardPandora                         11.9522       38.5495       81.3799       35.2308       15.7878        40     
produce:pandoraWriter:StandardPandora                  0.240664      0.384344      0.535643      0.378442      0.0640321       40     
produce:pandoraTrack:LArPandoraTrackCreation            1.73493       4.80511       10.1134       4.58363       1.72675        40     
produce:pandoraShower:LArPandoraShowerCreation          1.04967       5.42516       11.9319       4.73346       2.74949        40     
produce:pandoracalo:Calorimetry                        0.563663       2.20733       4.94374       1.95987      0.951851        40     
produce:pandoracalonosce:Calorimetry                   0.632094       2.02313       4.52771       1.7436       0.803567        40     
produce:pandorapid:Chi2ParticleID                     0.00309064    0.00419066    0.00725919    0.00403822    0.000798778      40     
produce:pandoraShowercalo:ShowerCalorimetry             1.23626       4.96384       13.282        4.79317       2.54913        40     
produce:pandoraShowercalonosce:ShowerCalorimetry        1.00913       4.64424       12.8877       3.94813       2.44536        40     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.161019      0.283192      0.451041      0.282964      0.0645085       40     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0533111     0.0974575     0.141637      0.0967226     0.0209805       40     
produce:emtrkmichelid:EmTrackMichelId                   37.2252       59.7214       80.9676       60.4269       11.506         40     
[art]:TriggerResults:TriggerResultInserter            4.3422e-05    8.58272e-05   0.000122051   8.80565e-05   1.71323e-05      40     
end_path:out1:RootOutput                               6.602e-06    1.45767e-05   3.2772e-05    1.4332e-05    4.61332e-06      40     
end_path:michelremoving:michelremoving                 0.0375624      0.16302      0.663748      0.0936781     0.152148        40     
end_path:michelremovinggnocchi:michelremoving          0.0351092     0.160231      0.674357      0.0898729     0.152809        40     
end_path:calibanagnocchi:CalibAnaTree                   13.8488       31.5606       77.0406       30.0676       13.9154        40     
end_path:out1:RootOutput(write)                         1.00182       1.58555       2.31476       1.58095      0.275237        40     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5073.18 MB
  Peak resident set size usage (VmHWM): 2663.74 MB
  Details saved in: 'mem.db'
====================================================================================================

{'art_out': 'pdhd_processing_1_418934_68_1750450442_stage1_stage2.root', 'tfile_out': 'pdhd_processing_1_418934_68_1750450442_stage1_stage2_tfile.root', 'start_time': 1750452127.896151, 'end_time': 1750461801.8098943}
justIN time: 2025-08-15 06:43:51 UTC       justIN version: 01.03.02