justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 80040.26@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID80040.26@justin-prod-sched02.dune.hep.ac.uk
Workflow ID3511
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-27 09:08:44
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw6
Last heartbeat2024-09-27 16:59:06
From worker nodeHostnamemh-7763-7.t2.ucsd.edu
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statestalled
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-27 14:25:02
Input fileshd-protodune:np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722.hdf5
Outputting started2024-09-27 16:24:13
Output files
Finished2024-09-27 17:36:31
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ailed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0152731 0.0217297 0.0301044 0.0528597 0.00347143 0.00661349 0.0107549 0.00676692 0.0139111 0.00564027 0.00236779 
----view 1: 1.89543e-05 1.90139e-05 1.89543e-05 1.90139e-05 1.90139e-05 1.89543e-05 1.90139e-05 1.89543e-05 1.90139e-05 1.89543e-05 1.90139e-05 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0217297 0.0301044 0.0528597 0.00347143 0.00661349 0.0107549 0.00676692 0.0139111 0.00564027 0.00236779 0.011292 
----view 1: 1.90139e-05 1.89543e-05 1.90139e-05 1.90139e-05 1.89543e-05 1.90139e-05 1.89543e-05 1.90139e-05 1.89543e-05 1.90139e-05 0.000293374 
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

27-Sep-2024 09:24:05 PDT  Closed output file "np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722_reco_stage1_reco_stage2_20240927T162405_keepup.root"
27-Sep-2024 09:24:05 PDT  Closed input file "np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              59.8957       98.6814       169.579       95.0843       30.2384        35     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000471227   0.00159194    0.00313723    0.00115277    0.000862736      35     
produce:fembfilter:PDHDFEMBFilter                      0.497285      0.516526      0.534646      0.518216     0.00849115       35     
produce:wclsdatahd:WireCellToolkit                      25.4801       26.6844       28.0247       26.7132      0.639012        35     
produce:gaushit:GausHitFinder                          0.387634      0.639255      0.928249      0.632154      0.144275        35     
produce:nhitsfilter:NumberOfHitsFilter                0.000214839   0.000311607   0.000485057   0.000311038   6.51579e-05      35     
produce:reco3d:SpacePointSolver                        0.246441       1.54007       5.70009       1.36447       1.18912        35     
produce:hitpdune:DisambigFromSpacePoints               0.292545      0.732258       1.40974      0.679023      0.336767        35     
produce:pandora:StandardPandora                         9.10547       30.6286       80.5459       24.0996       19.8608        35     
produce:pandoraWriter:StandardPandora                  0.0859141     0.138822      0.205272      0.138167      0.0344241       35     
produce:pandoraTrack:LArPandoraTrackCreation           0.755822       1.77732       3.54292       1.90159      0.720218        35     
produce:pandoraShower:LArPandoraShowerCreation         0.744152       1.88705       3.8271        2.00315       0.83609        35     
produce:pandoracalo:Calorimetry                         0.44212       0.95449       1.91355      0.936268      0.394725        35     
produce:pandoracalonosce:Calorimetry                   0.399937       0.90733       1.84668       0.88372      0.384419        35     
produce:pandorapid:Chi2ParticleID                      0.001201     0.00179695    0.00357106    0.00182175    0.00045009       35     
produce:pandoraShowercalo:ShowerCalorimetry            0.729908       1.8888        4.03685       1.88574       0.91119        35     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.699356       1.80255       3.77115       1.77799      0.864904        35     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0635802     0.108204       0.14986      0.101844      0.0230933       35     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0242639      0.03759      0.0524231     0.0358837    0.00834043       35     
produce:emtrkmichelid:EmTrackMichelId                   17.5485       26.7063       36.933        26.9564       5.46913        35     
[art]:TriggerResults:TriggerResultInserter             1.714e-05    2.57926e-05    8.071e-05    2.3679e-05    1.03755e-05      35     
end_path:out1:RootOutput                               4.329e-06    6.96283e-06    3.05e-05       5.8e-06     5.43978e-06      35     
end_path:michelremoving:michelremoving                 0.0137498     0.0702804     0.391425      0.026958      0.0873598       35     
end_path:michelremovinggnocchi:michelremoving          0.0134208     0.0671151     0.386884      0.0234347     0.0866451       35     
end_path:calibanagnocchi:CalibAnaTree                  0.327985      0.728304       1.46145      0.740966      0.307521        35     
end_path:out1:RootOutput(write)                        0.486829      0.756771       1.08653       0.78362      0.165147        35     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5131.73 MB
  Peak resident set size usage (VmHWM): 2743.54 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722_reco_stage1_reco_stage2_20240927T162405_keepup.root
\tHists: np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722_reco_stage1_reco_stage2_20240927T162405_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722_reco_stage1_reco_stage2_20240927T162405_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722_reco_stage1_reco_stage2_20240927T162405_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727450763.0,
    "core.end_time": 1727454246.0,
    "core.events": [
      29961,
      29965,
      29969,
      29973,
      29977,
      29981,
      29985,
      29989,
      29993,
      29997,
      30001,
      30005,
      30009,
      30013,
      30017,
      30021,
      30025,
      30029,
      30033,
      30037,
      30041,
      30045,
      30049,
      30053,
      30057,
      30061,
      30065,
      30069,
      30073,
      30077,
      30081,
      30085,
      30089,
      30093,
      30097
    ],
    "core.event_count": 35,
    "core.first_event_number": 29961,
    "core.last_event_number": 30097,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27334
    ],
    "core.runs_subruns": [
      2733400001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722_reco_stage1_reco_stage2_20240927T162405_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727450763.0,
    "core.end_time": 1727454246.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27334
    ],
    "core.runs_subruns": [
      2733400001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027334_0214_dataflow1_datawriter_0_20240621T044722.hdf5"
    }
  ]
}
justIN time: 2024-09-29 11:28:15 UTC       justIN version: 01.01.08