justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264378.106@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID264378.106@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3512
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-26 19:25:28
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw7
Last heartbeat2024-09-27 03:00:27
From worker nodeHostnamemh-7662-14.t2.ucsd.edu
cpuinfoAMD EPYC 7662 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-26 23:37:19
Input fileshd-protodune:np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451.hdf5
Outputting started2024-09-27 02:58:54
Output files
Finished2024-09-27 03:00:27
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

1 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40661 traces tagged "gauss"
FrameSaver: q=1.29454e+06 n=728884 tag=gauss
wclsFrameSaver: saving 52113 traces tagged "wiener"
FrameSaver: q=1.42096e+06 n=664345 tag=wiener
6076 hits in TPC 0
5376 hits in TPC 1
2734 hits in TPC 2
1025 hits in TPC 3
3009 hits in TPC 4
4164 hits in TPC 5
6071 hits in TPC 6
2114 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 24392 XUs and 19438 XVs -> 2385 XUVs
C:0 T:2 4920 XUs and 5867 XVs -> 119 XUVs
C:0 T:3 54 XUs and 63 XVs -> 0 XUVs
C:0 T:4 75 XUs and 119 XVs -> 5 XUVs
C:0 T:5 9289 XUs and 8516 XVs -> 366 XUVs
C:0 T:6 9292 XUs and 9030 XVs -> 263 XUVs
C:0 T:7 249 XUs and 281 XVs -> 19 XUVs
3157 XUVs total
967 collection wire objects
3157 potential space points
Neighbour search...
158141 tests to find 86508 neighbours
Iterating with no regularization...
Begin: 1.26096e+07
0 1.22476e+07
1 1.22194e+07
2 1.22184e+07
Now with regularization...
Begin: 1.18172e+07
0 1.18171e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

26-Sep-2024 19:58:44 PDT  Closed output file "np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451_reco_stage1_reco_stage2_20240927T025844_keepup.root"
26-Sep-2024 19:58:44 PDT  Closed input file "np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                               75.47        133.889       254.411       117.563       44.005         35     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000618536   0.00294158     0.005938     0.00278719    0.00174484       35     
produce:fembfilter:PDHDFEMBFilter                      0.702699      0.763622      0.850919      0.764853      0.0295084       35     
produce:wclsdatahd:WireCellToolkit                      35.4144       37.803        41.6374       37.5286       1.41163        35     
produce:gaushit:GausHitFinder                          0.608699       1.05412       1.66288       1.03059      0.267283        35     
produce:nhitsfilter:NumberOfHitsFilter                0.000297418   0.000454213   0.000739339   0.000441132   9.34644e-05      35     
produce:reco3d:SpacePointSolver                        0.414831       1.71357       5.18005       1.0236        1.29239        35     
produce:hitpdune:DisambigFromSpacePoints               0.353959       0.90522       2.20032      0.749035      0.461482        35     
produce:pandora:StandardPandora                         7.23063       41.0541       118.178       31.0332       26.7631        35     
produce:pandoraWriter:StandardPandora                  0.126992      0.214333      0.317994      0.197203      0.0541162       35     
produce:pandoraTrack:LArPandoraTrackCreation           0.843707       2.78501       7.1817        2.57318       1.41083        35     
produce:pandoraShower:LArPandoraShowerCreation         0.750937       2.77105       8.18626       2.39608       1.57356        35     
produce:pandoracalo:Calorimetry                        0.436629       1.23273       2.8816        1.06839      0.587902        35     
produce:pandoracalonosce:Calorimetry                   0.389318       1.16846       2.71463       1.0663       0.558223        35     
produce:pandorapid:Chi2ParticleID                     0.00186961    0.00266133     0.0038715    0.00255521    0.000562827      35     
produce:pandoraShowercalo:ShowerCalorimetry            0.710284       2.4334        6.40886       2.05333       1.39625        35     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.676966       2.25791       6.04358       1.87317       1.23607        35     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0803196     0.163363      0.235693      0.156076      0.0427449       35     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0282821     0.058232      0.0961076     0.0573357     0.0156909       35     
produce:emtrkmichelid:EmTrackMichelId                   22.306        35.0608       52.7796       34.5866       8.53313        35     
[art]:TriggerResults:TriggerResultInserter            2.0631e-05    3.42869e-05   8.8182e-05    2.8511e-05    1.41024e-05      35     
end_path:out1:RootOutput                               7.34e-06     1.06149e-05   2.4291e-05     9.451e-06    3.23503e-06      35     
end_path:michelremoving:michelremoving                 0.0201851     0.0782112     0.314185      0.0413232     0.0727619       35     
end_path:michelremovinggnocchi:michelremoving          0.0157534     0.0728878     0.307687      0.0337731     0.0721138       35     
end_path:calibanagnocchi:CalibAnaTree                  0.321978      0.988254       1.98659      0.885603      0.466012        35     
end_path:out1:RootOutput(write)                        0.714764       1.13923       1.83573       1.14285      0.284398        35     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5108.13 MB
  Peak resident set size usage (VmHWM): 2727.31 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451_reco_stage1_reco_stage2_20240927T025844_keepup.root
\tHists: np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451_reco_stage1_reco_stage2_20240927T025844_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451_reco_stage1_reco_stage2_20240927T025844_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451_reco_stage1_reco_stage2_20240927T025844_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727401195.0,
    "core.end_time": 1727405925.0,
    "core.events": [
      24644,
      24648,
      24652,
      24656,
      24660,
      24664,
      24668,
      24672,
      24676,
      24680,
      24684,
      24688,
      24692,
      24696,
      24700,
      24704,
      24708,
      24712,
      24716,
      24720,
      24724,
      24728,
      24732,
      24736,
      24740,
      24744,
      24748,
      24752,
      24756,
      24760,
      24764,
      24768,
      24772,
      24776,
      24780
    ],
    "core.event_count": 35,
    "core.first_event_number": 24644,
    "core.last_event_number": 24780,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27335
    ],
    "core.runs_subruns": [
      2733500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451_reco_stage1_reco_stage2_20240927T025844_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727401195.0,
    "core.end_time": 1727405925.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27335
    ],
    "core.runs_subruns": [
      2733500001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027335_0176_dataflow0_datawriter_0_20240621T061451.hdf5"
    }
  ]
}
justIN time: 2024-09-29 09:33:45 UTC       justIN version: 01.01.08