justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 276370.25@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID276370.25@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3697
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-10-17 10:33:39
SiteUS_BNL
EntryDUNE_US_BNL_sp02
Last heartbeat2024-10-17 13:30:20
From worker nodeHostnamespool0624.sdcc.bnl.gov
cpuinfoAMD EPYC 7351 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?False
Job statestalled
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-10-17 10:39:54
Input fileshd-protodune:np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953.hdf5
Outputting started2024-10-17 13:08:19
Output files
Finished2024-10-17 14:25:27
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ticks
wclsFrameSaver: saving 56364 traces tagged "gauss"
FrameSaver: q=2.00888e+06 n=1077464 tag=gauss
wclsFrameSaver: saving 72533 traces tagged "wiener"
FrameSaver: q=2.18764e+06 n=985417 tag=wiener
8221 hits in TPC 0
8353 hits in TPC 1
6824 hits in TPC 2
4137 hits in TPC 3
4333 hits in TPC 4
5160 hits in TPC 5
4733 hits in TPC 6
3019 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 22146 XUs and 16622 XVs -> 1059 XUVs
C:0 T:2 21371 XUs and 18442 XVs -> 564 XUVs
C:0 T:3 588 XUs and 510 XVs -> 12 XUVs
C:0 T:4 805 XUs and 661 XVs -> 36 XUVs
C:0 T:5 29978 XUs and 24569 XVs -> 787 XUVs
C:0 T:6 11013 XUs and 11597 XVs -> 350 XUVs
C:0 T:7 1639 XUs and 1021 XVs -> 68 XUVs
2876 XUVs total
1308 collection wire objects
2876 potential space points
Neighbour search...
61534 tests to find 33656 neighbours
Iterating with no regularization...
Begin: 4.92322e+07
0 4.4877e+07
1 4.47692e+07
2 4.47617e+07
Now with regularization...
Begin: 4.35638e+07
0 4.3562e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

17-Oct-2024 13:07:54 UTC  Closed output file "np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953_reco_stage1_reco_stage2_20241017T130754_keepup.root"
17-Oct-2024 13:07:54 UTC  Closed input file "np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              103.494       172.375       263.109       174.858       34.6193        35     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000720737   0.00309224    0.00847822    0.00335451    0.00171116       35     
produce:fembfilter:PDHDFEMBFilter                      0.803518       1.06452       2.57114      0.976938      0.324157        35     
produce:wclsdatahd:WireCellToolkit                      45.0885       49.3874       57.5477       48.9712       2.69946        35     
produce:gaushit:GausHitFinder                          0.613581       1.02423       1.56173       1.01546      0.222034        35     
produce:nhitsfilter:NumberOfHitsFilter                0.000345609   0.000523035   0.000721677   0.000517348   8.09965e-05      35     
produce:reco3d:SpacePointSolver                        0.408337       1.90486       5.91219       1.49717       1.22459        35     
produce:hitpdune:DisambigFromSpacePoints               0.396417       1.01433       1.8783       0.952391      0.372443        35     
produce:pandora:StandardPandora                         10.3633       36.2898       86.4519       33.2629       17.6031        35     
produce:pandoraWriter:StandardPandora                  0.132085      0.232749      0.378007      0.223145      0.0528381       35     
produce:pandoraTrack:LArPandoraTrackCreation            1.42363       3.23432       5.34464       2.99772       1.01075        35     
produce:pandoraShower:LArPandoraShowerCreation          1.26822       3.87977       7.44981       3.80826       1.41848        35     
produce:pandoracalo:Calorimetry                        0.772439       1.4507        2.76312       1.30853      0.498574        35     
produce:pandoracalonosce:Calorimetry                   0.742753       1.32614       2.75256       1.17913      0.484867        35     
produce:pandorapid:Chi2ParticleID                     0.00170758    0.00290403    0.00530792    0.00271911    0.000895422      35     
produce:pandoraShowercalo:ShowerCalorimetry             1.39045       3.14982       6.83451       2.80424       1.34706        35     
produce:pandoraShowercalonosce:ShowerCalorimetry        1.15164       3.05423       6.66208       2.65812       1.29887        35     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.114953      0.183544      0.352839      0.179851      0.0463521       35     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0405736     0.0628834     0.122279      0.0611277     0.0164064       35     
produce:emtrkmichelid:EmTrackMichelId                   37.265        62.5331       91.9399       62.3841       10.9519        35     
[art]:TriggerResults:TriggerResultInserter            2.5439e-05    5.09654e-05    8.843e-05    5.0079e-05    1.62115e-05      35     
end_path:out1:RootOutput                               9.24e-06     1.54936e-05   8.2829e-05     1.128e-05    1.23399e-05      35     
end_path:michelremoving:michelremoving                 0.0245933     0.102033      0.419409      0.0453849     0.101619        35     
end_path:michelremovinggnocchi:michelremoving          0.0228852     0.096698      0.346753      0.0417837     0.0932113       35     
end_path:calibanagnocchi:CalibAnaTree                  0.403446       1.02254       1.99906      0.919235      0.361425        35     
end_path:out1:RootOutput(write)                        0.699417       1.17879       1.63246       1.17605      0.203341        35     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5031.48 MB
  Peak resident set size usage (VmHWM): 2370.91 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953_reco_stage1_reco_stage2_20241017T130754_keepup.root
\tHists: np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953_reco_stage1_reco_stage2_20241017T130754_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953_reco_stage1_reco_stage2_20241017T130754_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953_reco_stage1_reco_stage2_20241017T130754_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1729164380.0,
    "core.end_time": 1729170479.0,
    "core.events": [
      31641,
      31645,
      31649,
      31653,
      31657,
      31661,
      31665,
      31669,
      31673,
      31677,
      31681,
      31685,
      31689,
      31693,
      31697,
      31701,
      31705,
      31709,
      31713,
      31717,
      31721,
      31725,
      31729,
      31733,
      31737,
      31741,
      31745,
      31749,
      31753,
      31757,
      31761,
      31765,
      31769,
      31773,
      31777
    ],
    "core.event_count": 35,
    "core.first_event_number": 31641,
    "core.last_event_number": 31777,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27404
    ],
    "core.runs_subruns": [
      2740400001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953_reco_stage1_reco_stage2_20241017T130754_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1729164380.0,
    "core.end_time": 1729170479.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27404
    ],
    "core.runs_subruns": [
      2740400001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027404_0226_dataflow1_datawriter_0_20240624T120953.hdf5"
    }
  ]
}
justIN time: 2024-11-21 19:52:33 UTC       justIN version: 01.01.09