justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 270270.103@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID270270.103@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3594
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-10-05 19:01:15
SiteUS_BNL
EntryDUNE_US_BNL_sp02
Last heartbeat2024-10-06 00:34:16
From worker nodeHostnamespool0589.sdcc.bnl.gov
cpuinfoAMD EPYC 7351 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?False
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-10-05 22:59:24
Input fileshd-protodune:np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016.hdf5
Outputting started2024-10-06 00:31:40
Output files
Finished2024-10-06 00:34:16
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

d 753104 neighbours
Iterating with no regularization...
Begin: 9.69002e+07
0 7.52676e+07
1 7.35489e+07
2 7.33029e+07
3 7.32339e+07
Now with regularization...
Begin: 6.80448e+07
0 6.79406e+07
1 6.79178e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

Begin processing the 31st record. run: 27937 subRun: 1 event: 226916 at 06-Oct-2024 00:30:20 UTC
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 112036 traces, 112036 summary
Retagger: tagging trace set: gauss with 79771 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 79771 traces tagged "gauss"
FrameSaver: q=2.86807e+06 n=1367183 tag=gauss
wclsFrameSaver: saving 112036 traces tagged "wiener"
FrameSaver: q=3.15955e+06 n=1220106 tag=wiener
4258 hits in TPC 0
3637 hits in TPC 1
20179 hits in TPC 2
06-Oct-2024 00:31:17 UTC  Closed output file "np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016_reco_stage1_reco_stage2_20241006T003117_keepup.root"
06-Oct-2024 00:31:17 UTC  Closed input file "np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              50.223        133.365       218.602       129.948       31.055         31     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000466471   0.00271798     0.0047091     0.0031482    0.00123933       31     
produce:fembfilter:PDHDFEMBFilter                      0.691037      0.817652       1.63775      0.762121      0.186082        31     
produce:wclsdatahd:WireCellToolkit                      43.2312       45.1681       47.6084       45.0481       1.05717        31     
produce:gaushit:GausHitFinder                          0.578194      0.844228       1.41266      0.821982      0.178914        31     
produce:nhitsfilter:NumberOfHitsFilter                0.000353541   0.000600797    0.0034343    0.00050674    0.000524036      31     
produce:reco3d:SpacePointSolver                        0.267346       3.9668        16.1301       3.04437       3.58858        30     
produce:hitpdune:DisambigFromSpacePoints               0.274505      0.799785       1.73657      0.710367      0.394649        30     
produce:pandora:StandardPandora                         9.49687       21.101        66.3811       18.1611       11.7595        30     
produce:pandoraWriter:StandardPandora                  0.125231      0.184922      0.248411      0.180327      0.0344295       30     
produce:pandoraTrack:LArPandoraTrackCreation           0.925666       2.22615       3.62357       2.07511      0.761848        30     
produce:pandoraShower:LArPandoraShowerCreation          1.0736        2.76172       5.53881       2.49783       1.21064        30     
produce:pandoracalo:Calorimetry                        0.466682      0.976015       1.74077      0.917431      0.318248        30     
produce:pandoracalonosce:Calorimetry                   0.423526      0.943956       1.72822      0.855244      0.344878        30     
produce:pandorapid:Chi2ParticleID                     0.00141945    0.00217631    0.00449195     0.0020831    0.000554062      30     
produce:pandoraShowercalo:ShowerCalorimetry             1.05072       2.2241        4.98001       1.93578      0.955363        30     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.999122       2.12764       4.9769        1.91389      0.935169        30     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0999381     0.159391      0.269194      0.157294      0.0332408       30     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0327274     0.0551707     0.094049      0.0537299     0.0130865       30     
produce:emtrkmichelid:EmTrackMichelId                   31.8676       49.8557       68.7294       49.2681       9.32141        30     
[art]:TriggerResults:TriggerResultInserter             2.018e-05    3.35049e-05   0.00011883     2.759e-05    1.82047e-05      31     
end_path:out1:RootOutput                               6.72e-06     9.71419e-06    3.988e-05      7.9e-06     6.15785e-06      31     
end_path:michelremoving:michelremoving                 3.544e-05     0.075809      0.348214      0.0372438     0.0746877       31     
end_path:michelremovinggnocchi:michelremoving          1.221e-05     0.0710061     0.358083      0.0346121     0.0739137       31     
end_path:calibanagnocchi:CalibAnaTree                  1.198e-05     0.679146       1.23043      0.658152      0.256266        31     
end_path:out1:RootOutput(write)                        0.648017       1.00181       1.37945       1.00879      0.199314        31     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4965.48 MB
  Peak resident set size usage (VmHWM): 2082.11 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016_reco_stage1_reco_stage2_20241006T003117_keepup.root
\tHists: np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016_reco_stage1_reco_stage2_20241006T003117_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016_reco_stage1_reco_stage2_20241006T003117_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016_reco_stage1_reco_stage2_20241006T003117_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1728170485.0,
    "core.end_time": 1728174686.0,
    "core.events": [
      226676,
      226684,
      226692,
      226700,
      226708,
      226716,
      226724,
      226732,
      226740,
      226748,
      226756,
      226764,
      226772,
      226780,
      226788,
      226796,
      226804,
      226812,
      226820,
      226828,
      226836,
      226844,
      226852,
      226860,
      226868,
      226876,
      226884,
      226892,
      226900,
      226908,
      226916
    ],
    "core.event_count": 31,
    "core.first_event_number": 226676,
    "core.last_event_number": 226916,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27937
    ],
    "core.runs_subruns": [
      2793700001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016_reco_stage1_reco_stage2_20241006T003117_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1728170485.0,
    "core.end_time": 1728174686.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27937
    ],
    "core.runs_subruns": [
      2793700001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027937_0914_dataflow4_datawriter_0_20240710T025016.hdf5"
    }
  ]
}
justIN time: 2024-11-21 20:04:49 UTC       justIN version: 01.01.09