justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 79982.120@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID79982.120@justin-prod-sched02.dune.hep.ac.uk
Workflow ID3518
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-27 08:36:24
SiteUS_UCSD
EntryCMSHTPC_T2_US_UCSD_gw6
Last heartbeat2024-09-27 16:41:02
From worker nodeHostnamemh-7662-6.t2.ucsd.edu
cpuinfoAMD EPYC 7662 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statestalled
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-27 14:20:51
Input fileshd-protodune:np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422.hdf5
Outputting started2024-09-27 16:06:27
Output files
Finished2024-09-27 17:36:31
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

1949 0.0319672 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0 0 0 0 0 0 0 0 0 0 0 
----view 1: 0.0343628 0.0331268 0.0257263 0.00411415 0.002491 0.00456429 0.030838 0.0443115 0.0341949 0.0319672 0.0115509 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000433326 0.00243831 0.00140363 0.000996053 0.00409913 0.000729501 0.00051409 8.89301e-05 0.000926375 0.000355005 0.00109285 3.46899e-05 2.7895e-05 
----view 1: 0.000131726 0.000131786 2.83122e-05 2.60472e-05 2.61068e-05 2.61068e-05 2.60472e-05 2.61068e-05 2.60472e-05 2.61068e-05 2.61068e-05 2.60472e-05 2.61068e-05 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00243831 0.00140363 0.000996053 0.00409913 0.000729501 0.00051409 8.89301e-05 0.000926375 0.000355005 0.00109285 3.46899e-05 2.7895e-05 1.54376e-05 
----view 1: 0.000131786 2.83122e-05 2.60472e-05 2.61068e-05 2.61068e-05 2.60472e-05 2.61068e-05 2.60472e-05 2.61068e-05 2.61068e-05 2.60472e-05 2.61068e-05 2.61068e-05 
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

27-Sep-2024 09:06:17 PDT  Closed output file "np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422_reco_stage1_reco_stage2_20240927T160617_keepup.root"
27-Sep-2024 09:06:17 PDT  Closed input file "np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              60.3449       112.242       211.888       97.7543       38.0745        35     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000885466   0.00258398    0.00535437    0.00238044    0.00121449       35     
produce:fembfilter:PDHDFEMBFilter                      0.687266      0.729841      0.787063      0.731272      0.0237024       35     
produce:wclsdatahd:WireCellToolkit                      34.2312       38.4428       46.2159       37.0723       3.29263        35     
produce:gaushit:GausHitFinder                          0.387506      0.810535       1.38666      0.765467      0.229149        35     
produce:nhitsfilter:NumberOfHitsFilter                0.000244677   0.000423402   0.000738041   0.000410412   0.000100553      35     
produce:reco3d:SpacePointSolver                         0.15781       1.28837       4.72578       1.09126       1.03074        35     
produce:hitpdune:DisambigFromSpacePoints               0.185035      0.713868       1.98135      0.625235      0.416843        35     
produce:pandora:StandardPandora                         4.81335       27.6215       82.199        19.6708       20.8665        35     
produce:pandoraWriter:StandardPandora                  0.0824581     0.187349      0.332929      0.171134      0.0571049       35     
produce:pandoraTrack:LArPandoraTrackCreation           0.438256       2.12647       6.46306       1.75236       1.38765        35     
produce:pandoraShower:LArPandoraShowerCreation         0.390962       2.16812       6.67323       1.53599       1.52271        35     
produce:pandoracalo:Calorimetry                        0.230455      0.976051       2.75572      0.801223      0.581177        35     
produce:pandoracalonosce:Calorimetry                   0.197247      0.910457       2.64696      0.749205      0.557194        35     
produce:pandorapid:Chi2ParticleID                     0.00119947    0.00222373    0.00359597    0.00213687    0.000577364      35     
produce:pandoraShowercalo:ShowerCalorimetry            0.335869       1.81915       5.48554       1.38064       1.20844        35     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.328288       1.71773       5.02646       1.27845       1.11131        35     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0821654      0.15234      0.268963      0.144169      0.0444016       35     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0274806     0.0525142     0.107414      0.0520964     0.0160526       35     
produce:emtrkmichelid:EmTrackMichelId                   14.6277       30.5537       58.898        28.451        9.14305        35     
[art]:TriggerResults:TriggerResultInserter            1.7491e-05    2.93989e-05   8.5223e-05    2.4561e-05    1.34962e-05      35     
end_path:out1:RootOutput                               8.11e-06     1.30153e-05   6.3762e-05     9.46e-06     1.26823e-05      35     
end_path:michelremoving:michelremoving                 0.0170724     0.0690141     0.282636      0.0322295     0.0734835       35     
end_path:michelremovinggnocchi:michelremoving          0.0166651     0.067231      0.277461      0.0292643     0.0726482       35     
end_path:calibanagnocchi:CalibAnaTree                  0.178125      0.720213       2.20479      0.575333      0.433325        35     
end_path:out1:RootOutput(write)                         0.48264      0.969529       1.67813       0.91374      0.273288        35     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5046.92 MB
  Peak resident set size usage (VmHWM): 2703.65 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422_reco_stage1_reco_stage2_20240927T160617_keepup.root
\tHists: np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422_reco_stage1_reco_stage2_20240927T160617_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422_reco_stage1_reco_stage2_20240927T160617_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422_reco_stage1_reco_stage2_20240927T160617_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727449207.0,
    "core.end_time": 1727453178.0,
    "core.events": [
      2663,
      2667,
      2671,
      2675,
      2679,
      2683,
      2687,
      2691,
      2695,
      2699,
      2703,
      2707,
      2711,
      2715,
      2719,
      2723,
      2727,
      2731,
      2735,
      2739,
      2743,
      2747,
      2751,
      2755,
      2759,
      2763,
      2767,
      2771,
      2775,
      2779,
      2783,
      2787,
      2791,
      2795,
      2799
    ],
    "core.event_count": 35,
    "core.first_event_number": 2663,
    "core.last_event_number": 2799,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27408
    ],
    "core.runs_subruns": [
      2740800001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422_reco_stage1_reco_stage2_20240927T160617_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727449207.0,
    "core.end_time": 1727453178.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27408
    ],
    "core.runs_subruns": [
      2740800001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027408_0019_dataflow3_datawriter_0_20240624T154422.hdf5"
    }
  ]
}
justIN time: 2024-09-29 11:28:05 UTC       justIN version: 01.01.08