justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 59401.165@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID59401.165@justin-prod-sched02.dune.hep.ac.uk
Workflow ID2867
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-08-18 19:23:48
SiteNL_NIKHEF
EntryVIRGO_NL_NIKHEF_juk
Last heartbeat2024-08-18 21:56:45
From worker nodeHostnamewn-choc-039.farm.nikhef.nl
cpuinfoIntel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit129600 (36 hours)
Inner Apptainer?True
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-08-18 20:22:19
Input fileshd-protodune:np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404.hdf5
Outputting started2024-08-18 21:55:37
Output files
Finished2024-08-18 21:56:45
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

08 XUVs
C:0 T:6 37525 XUs and 29991 XVs -> 1161 XUVs
C:0 T:7 621 XUs and 881 XVs -> 36 XUVs
2447 XUVs total
1261 collection wire objects
2447 potential space points
Neighbour search...
83257 tests to find 47020 neighbours
Iterating with no regularization...
Begin: 5.61357e+06
0 5.23491e+06
1 5.21683e+06
2 5.21465e+06
Now with regularization...
Begin: 4.92361e+06
0 4.92292e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

no of hits closeby  2   event 169680 TrkackID 12 startx, y, z 366.108 510.481 331.251  wireno, peakt tpcno 478 4419.93 6 dist 0min T, max_T3828 4466.76
no of hits closeby  1   event 169680 TrkackID 24 startx, y, z 34.9133 591.936 442.229  wireno, peakt tpcno 381 5332.05 5 dist 0min T, max_T5325 5814.42
no of hits closeby  2   event 169680 TrkackID 12 startx, y, z 366.108 510.481 331.251  wireno, peakt tpcno 478 4419.93 6 dist 0min T, max_T3828 4466.76
no of hits closeby  1   event 169680 TrkackID 24 startx, y, z 34.9133 591.936 442.229  wireno, peakt tpcno 381 5332.05 5 dist 0min T, max_T5325 5814.42
18-Aug-2024 23:55:28 CEST  Closed output file "np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404_reco_stage1_reco_stage2_20240818T215528_keepup.root"
18-Aug-2024 23:55:28 CEST  Closed input file "np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              91.7791       139.755       266.027       132.508       34.4128        30     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00061557    0.00268115    0.00474053    0.00277331    0.00121242       30     
produce:fembfilter:PDHDFEMBFilter                      0.704196      0.717499      0.762217      0.714284      0.0123987       30     
produce:wclsdatahd:WireCellToolkit                      47.5852       50.3996       56.8358       49.5217       2.47935        30     
produce:gaushit:GausHitFinder                          0.680773       1.01377       1.64229       1.01666      0.203781        30     
produce:nhitsfilter:NumberOfHitsFilter                0.000360436   0.000481444   0.00076912    0.000468529   9.25194e-05      30     
produce:reco3d:SpacePointSolver                        0.485308       1.75052       7.75175       1.19061       1.69605        30     
produce:hitpdune:DisambigFromSpacePoints               0.437449      0.917637       2.72675      0.805865       0.47286        30     
produce:pandora:StandardPandora                         10.5336       32.0542       93.1705       25.0091       17.5992        30     
produce:pandoraWriter:StandardPandora                  0.146393      0.215428      0.361566      0.208877      0.0443853       30     
produce:pandoraTrack:LArPandoraTrackCreation            1.71751       3.76828       8.77852       3.56358       1.73629        30     
produce:pandoraShower:LArPandoraShowerCreation          1.08671       4.55485       11.8376       4.17047       2.31148        30     
produce:pandoracalo:Calorimetry                        0.610097       1.53272       3.66954       1.45063      0.696126        30     
produce:pandoracalonosce:Calorimetry                   0.580596       1.47009       3.73499       1.36302      0.709137        30     
produce:pandorapid:Chi2ParticleID                     0.00177028    0.00256917    0.00466681    0.00242001    0.000627108      30     
produce:pandoraShowercalo:ShowerCalorimetry            0.758172       3.36075       8.97599       3.11891       1.79138        30     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.681696       3.22345       8.52042       3.05288       1.67181        30     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.076129      0.171134      0.238436       0.18302      0.0370133       30     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0282622      0.06497       0.11155      0.0682551     0.0153452       30     
produce:emtrkmichelid:EmTrackMichelId                   20.0619       31.9581       56.2646       30.6992       7.13261        30     
[art]:TriggerResults:TriggerResultInserter            3.2333e-05    5.6904e-05    0.000115448   5.6706e-05    1.33325e-05      30     
end_path:out1:RootOutput                               8.336e-06    1.3641e-05    3.1883e-05    1.33115e-05   4.84191e-06      30     
end_path:michelremoving:michelremoving                 0.0225589     0.124819      0.420276      0.0572335     0.113494        30     
end_path:michelremovinggnocchi:michelremoving          0.0201588     0.119572      0.407949      0.057619      0.111336        30     
end_path:calibanagnocchi:CalibAnaTree                  0.432801       1.13987       2.86944       1.05271      0.551356        30     
end_path:out1:RootOutput(write)                        0.622198       1.01658       1.6853        1.02042      0.221198        30     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4961.51 MB
  Peak resident set size usage (VmHWM): 2645.7 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404_reco_stage1_reco_stage2_20240818T215528_keepup.root
\tHists: np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404_reco_stage1_reco_stage2_20240818T215528_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404_reco_stage1_reco_stage2_20240818T215528_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404_reco_stage1_reco_stage2_20240818T215528_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1724013885.0,
    "core.end_time": 1724018128.0,
    "core.events": [
      169448,
      169456,
      169464,
      169472,
      169480,
      169488,
      169496,
      169504,
      169512,
      169520,
      169528,
      169536,
      169544,
      169552,
      169560,
      169568,
      169576,
      169584,
      169592,
      169600,
      169608,
      169616,
      169624,
      169632,
      169640,
      169648,
      169656,
      169664,
      169672,
      169680
    ],
    "core.event_count": 30,
    "core.first_event_number": 169448,
    "core.last_event_number": 169680,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      28762
    ],
    "core.runs_subruns": [
      2876200001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404_reco_stage1_reco_stage2_20240818T215528_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1724013885.0,
    "core.end_time": 1724018128.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      28762
    ],
    "core.runs_subruns": [
      2876200001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run028762_0706_dataflow0_datawriter_0_20240818T191404.hdf5"
    }
  ]
}
justIN time: 2024-11-17 05:45:00 UTC       justIN version: 01.01.09