justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 257751.33@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID257751.33@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3264
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-16 18:11:39
SiteUK_Edinburgh
EntryDUNE_UK_SGridECDF_ce1_multicore
Last heartbeat2024-09-16 19:43:29
From worker nodeHostnamenode1g16.ecdf.ed.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-16 18:13:24
Input fileshd-protodune:np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617.hdf5
Outputting started2024-09-16 19:42:07
Output files
Finished2024-09-16 19:43:29
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

o 5859 ticks
wclsFrameSaver: saving 56598 traces tagged "gauss"
FrameSaver: q=1.24728e+06 n=1055057 tag=gauss
wclsFrameSaver: saving 74440 traces tagged "wiener"
FrameSaver: q=1.3489e+06 n=965564 tag=wiener
5831 hits in TPC 0
4836 hits in TPC 1
4362 hits in TPC 2
2679 hits in TPC 3
4821 hits in TPC 4
6910 hits in TPC 5
6023 hits in TPC 6
3700 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 6659 XUs and 4554 XVs -> 118 XUVs
C:0 T:2 18015 XUs and 16471 XVs -> 461 XUVs
C:0 T:3 20 XUs and 14 XVs -> 0 XUVs
C:0 T:4 227 XUs and 224 XVs -> 17 XUVs
C:0 T:5 19413 XUs and 20549 XVs -> 2677 XUVs
C:0 T:6 40574 XUs and 32448 XVs -> 1034 XUVs
C:0 T:7 854 XUs and 1267 XVs -> 57 XUVs
4364 XUVs total
1305 collection wire objects
4364 potential space points
Neighbour search...
196862 tests to find 121620 neighbours
Iterating with no regularization...
Begin: 9.81909e+06
0 8.91746e+06
1 8.8488e+06
2 8.83522e+06
3 8.83121e+06
Now with regularization...
Begin: 8.24883e+06
0 8.24605e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

16-Sep-2024 20:41:55 BST  Closed output file "np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617_reco_stage1_reco_stage2_20240916T194155_keepup.root"
16-Sep-2024 20:41:55 BST  Closed input file "np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              79.2354       108.861       136.306       111.066       15.9969        31     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000415903   0.00154238    0.00389494    0.00136569    0.000924937      31     
produce:fembfilter:PDHDFEMBFilter                      0.540495      0.912577       1.08558       1.00613      0.162382        31     
produce:wclsdatahd:WireCellToolkit                      37.6173       56.4964       60.2577       58.3916       4.98041        31     
produce:gaushit:GausHitFinder                          0.454066      0.938386       1.40812      0.922426      0.263729        31     
produce:nhitsfilter:NumberOfHitsFilter                0.000330043   0.000495456   0.00101693    0.00044456    0.000139701      31     
produce:reco3d:SpacePointSolver                        0.228256      0.957215       3.5634       0.757572      0.758464        31     
produce:hitpdune:DisambigFromSpacePoints               0.166873      0.656241       1.25434      0.550034       0.27551        31     
produce:pandora:StandardPandora                          4.751        15.2606       28.3569       15.662        5.48628        31     
produce:pandoraWriter:StandardPandora                  0.0864875     0.203096      0.392369      0.184173      0.0600624       31     
produce:pandoraTrack:LArPandoraTrackCreation            0.43098       2.16243       4.55662       2.01496      0.910141        31     
produce:pandoraShower:LArPandoraShowerCreation         0.509514       2.29191       5.53921       1.91465       1.15611        31     
produce:pandoracalo:Calorimetry                        0.240121      0.936564       1.74142      0.921606      0.367012        31     
produce:pandoracalonosce:Calorimetry                   0.208855      0.868684       1.5508       0.845984      0.360333        31     
produce:pandorapid:Chi2ParticleID                     0.00124314    0.00226152    0.00305498    0.00226915    0.000518851      31     
produce:pandoraShowercalo:ShowerCalorimetry            0.405242       2.05221       3.98532       1.87454       1.00826        31     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.374624       2.00781       3.98156       1.89184      0.950764        31     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0775849     0.163653      0.256123      0.149132      0.0540756       31     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0300406     0.0634854     0.101733      0.0573436     0.021101        31     
produce:emtrkmichelid:EmTrackMichelId                   12.8635       20.722        30.2811       21.3994       4.07574        31     
[art]:TriggerResults:TriggerResultInserter            2.6711e-05    6.02128e-05   7.8942e-05     6.25e-05     1.08993e-05      31     
end_path:out1:RootOutput                               7.838e-06    1.35808e-05   1.8022e-05    1.3571e-05    2.72869e-06      31     
end_path:michelremoving:michelremoving                 0.0232505     0.119728      0.421198      0.0678108      0.10449        31     
end_path:michelremovinggnocchi:michelremoving          0.0198197     0.117678      0.439373      0.0729284     0.106026        31     
end_path:calibanagnocchi:CalibAnaTree                  0.170338      0.725896       1.11498      0.745381      0.262494        31     
end_path:out1:RootOutput(write)                        0.467812      0.989875       1.38126      0.966585      0.221591        31     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5797.03 MB
  Peak resident set size usage (VmHWM): 2895.85 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617_reco_stage1_reco_stage2_20240916T194155_keepup.root
\tHists: np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617_reco_stage1_reco_stage2_20240916T194155_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617_reco_stage1_reco_stage2_20240916T194155_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617_reco_stage1_reco_stage2_20240916T194155_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1726512305.0,
    "core.end_time": 1726515716.0,
    "core.events": [
      60039,
      60047,
      60063,
      60071,
      60079,
      60087,
      60095,
      60103,
      60111,
      60119,
      60127,
      60135,
      60143,
      60151,
      60159,
      60167,
      60175,
      60183,
      60191,
      60199,
      60207,
      60215,
      60223,
      60231,
      60239,
      60247,
      60255,
      60263,
      60271,
      60279,
      60287
    ],
    "core.event_count": 31,
    "core.first_event_number": 60039,
    "core.last_event_number": 60287,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      29197
    ],
    "core.runs_subruns": [
      2919700001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617_reco_stage1_reco_stage2_20240916T194155_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1726512305.0,
    "core.end_time": 1726515716.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      29197
    ],
    "core.runs_subruns": [
      2919700001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run029197_0242_dataflow7_datawriter_0_20240916T173617.hdf5"
    }
  ]
}
justIN time: 2024-09-29 09:37:35 UTC       justIN version: 01.01.08