justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264556.68@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID264556.68@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3517
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-27 02:18:48
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw6
Last heartbeat2024-09-27 07:22:55
From worker nodeHostnamemh-7662-20.t2.ucsd.edu
cpuinfoAMD EPYC 7662 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-27 04:43:25
Input fileshd-protodune:np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625.hdf5
Outputting started2024-09-27 07:17:39
Output files
Finished2024-09-27 07:22:55
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47275 traces tagged "gauss"
FrameSaver: q=1.74223e+06 n=873430 tag=gauss
wclsFrameSaver: saving 62723 traces tagged "wiener"
FrameSaver: q=1.88233e+06 n=790784 tag=wiener
3396 hits in TPC 0
3751 hits in TPC 1
4011 hits in TPC 2
2282 hits in TPC 3
7492 hits in TPC 4
9232 hits in TPC 5
3437 hits in TPC 6
1776 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3659 XUs and 5366 XVs -> 58 XUVs
C:0 T:2 9351 XUs and 8089 XVs -> 375 XUVs
C:0 T:3 5 XUs and 5 XVs -> 0 XUVs
C:0 T:4 294 XUs and 422 XVs -> 12 XUVs
C:0 T:5 53039 XUs and 100457 XVs -> 3026 XUVs
C:0 T:6 5058 XUs and 6743 XVs -> 126 XUVs
C:0 T:7 520 XUs and 503 XVs -> 10 XUVs
3607 XUVs total
1213 collection wire objects
3607 potential space points
Neighbour search...
223581 tests to find 106012 neighbours
Iterating with no regularization...
Begin: 4.15441e+07
0 3.34846e+07
1 3.28004e+07
2 3.27223e+07
3 3.27098e+07
Now with regularization...
Begin: 3.18264e+07
0 3.18206e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

27-Sep-2024 00:17:28 PDT  Closed output file "np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625_reco_stage1_reco_stage2_20240927T071728_keepup.root"
27-Sep-2024 00:17:28 PDT  Closed input file "np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              64.2341       100.373       163.78        99.3484       20.3244        35     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000643224   0.00292659    0.00585931     0.0030652    0.00133297       35     
produce:fembfilter:PDHDFEMBFilter                      0.710434      0.764586       1.03223       0.75551      0.0541155       35     
produce:wclsdatahd:WireCellToolkit                      35.0675       38.1127       49.5994       37.5534       2.40003        35     
produce:gaushit:GausHitFinder                          0.515637      0.874104       1.34419      0.864437      0.178413        35     
produce:nhitsfilter:NumberOfHitsFilter                0.000287142   0.000433128   0.000647031   0.000428488   8.18899e-05      35     
produce:reco3d:SpacePointSolver                        0.212741       1.20617       4.20463       1.02936      0.802573        35     
produce:hitpdune:DisambigFromSpacePoints               0.174962      0.650515       2.15572      0.579759      0.357481        35     
produce:pandora:StandardPandora                         6.80023       18.1843       46.5739       16.6368       7.84941        35     
produce:pandoraWriter:StandardPandora                  0.113154      0.183386      0.274658       0.18613      0.0375074       35     
produce:pandoraTrack:LArPandoraTrackCreation           0.503666       1.7053        4.1417        1.62069      0.808502        35     
produce:pandoraShower:LArPandoraShowerCreation         0.648063       2.04733       5.64665       1.74478       1.12517        35     
produce:pandoracalo:Calorimetry                        0.316259      0.827738       1.57291       0.78761      0.320493        35     
produce:pandoracalonosce:Calorimetry                   0.297957      0.756985       1.48938       0.73453      0.295977        35     
produce:pandorapid:Chi2ParticleID                     0.00130312    0.00227719     0.0041793    0.00227552    0.000628655      35     
produce:pandoraShowercalo:ShowerCalorimetry             0.52999       1.73239       4.49822       1.55852      0.827362        35     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.524062        1.651        4.67612       1.52939      0.816894        35     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0801428     0.155784      0.247107      0.159544      0.0364856       35     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0292194     0.0536264     0.0809143     0.0539047     0.0114959       35     
produce:emtrkmichelid:EmTrackMichelId                   16.3293       29.5709       46.5068       29.0844       6.73262        35     
[art]:TriggerResults:TriggerResultInserter            1.8766e-05    4.00499e-05   0.000127069   3.4405e-05    2.26794e-05      35     
end_path:out1:RootOutput                               1.061e-05    1.27809e-05   2.5268e-05    1.1862e-05    2.74958e-06      35     
end_path:michelremoving:michelremoving                 0.0161089     0.0828173     0.400472      0.0419458     0.083379        35     
end_path:michelremovinggnocchi:michelremoving          0.015123      0.0799962     0.405822      0.0341133     0.0863108       35     
end_path:calibanagnocchi:CalibAnaTree                  0.230766      0.596103       1.34686      0.554675      0.255534        35     
end_path:out1:RootOutput(write)                        0.541901      0.979194       1.51189      0.948733      0.233039        35     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5022.62 MB
  Peak resident set size usage (VmHWM): 2643.27 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625_reco_stage1_reco_stage2_20240927T071728_keepup.root
\tHists: np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625_reco_stage1_reco_stage2_20240927T071728_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625_reco_stage1_reco_stage2_20240927T071728_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625_reco_stage1_reco_stage2_20240927T071728_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727417893.0,
    "core.end_time": 1727421448.0,
    "core.events": [
      95764,
      95768,
      95772,
      95776,
      95780,
      95784,
      95788,
      95792,
      95796,
      95800,
      95804,
      95808,
      95812,
      95816,
      95820,
      95824,
      95828,
      95832,
      95836,
      95840,
      95844,
      95848,
      95852,
      95856,
      95860,
      95864,
      95868,
      95872,
      95876,
      95880,
      95884,
      95888,
      95892,
      95896,
      95900
    ],
    "core.event_count": 35,
    "core.first_event_number": 95764,
    "core.last_event_number": 95900,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27394
    ],
    "core.runs_subruns": [
      2739400001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625_reco_stage1_reco_stage2_20240927T071728_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727417893.0,
    "core.end_time": 1727421448.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27394
    ],
    "core.runs_subruns": [
      2739400001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027394_0684_dataflow0_datawriter_0_20240623T221625.hdf5"
    }
  ]
}
justIN time: 2024-09-29 09:38:54 UTC       justIN version: 01.01.08