justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 264654.119@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID264654.119@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3519
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-27 04:41:09
SiteCA_SFU
EntryDUNE_CA_SFU_lcg-ce3
Last heartbeat2024-09-27 17:26:58
From worker nodeHostnamecdr268.int.cedar.computecanada.ca
cpuinfoIntel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit84598 (23 hours)
Inner Apptainer?True
Job statestalled
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-27 06:29:10
Input fileshd-protodune:np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620.hdf5
Outputting started2024-09-27 17:20:05
Output files
Finished2024-09-27 17:36:32
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

=734773 tag=wiener
3096 hits in TPC 0
2207 hits in TPC 1
6931 hits in TPC 2
3944 hits in TPC 3
2178 hits in TPC 4
4478 hits in TPC 5
8017 hits in TPC 6
5071 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 612 XUs and 425 XVs -> 8 XUVs
C:0 T:2 49673 XUs and 106153 XVs -> 1384 XUVs
C:0 T:3 418 XUs and 286 XVs -> 10 XUVs
C:0 T:4 112 XUs and 158 XVs -> 15 XUVs
C:0 T:5 12898 XUs and 5725 XVs -> 134 XUVs
C:0 T:6 41515 XUs and 55570 XVs -> 1402 XUVs
C:0 T:7 167 XUs and 171 XVs -> 5 XUVs
2958 XUVs total
1396 collection wire objects
2958 potential space points
Neighbour search...
100172 tests to find 57176 neighbours
Iterating with no regularization...
Begin: 2.03405e+07
0 1.74903e+07
1 1.73022e+07
2 1.72851e+07
Now with regularization...
Begin: 1.62303e+07
0 1.6225e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

no of hits closeby  0   event 10676 TrkackID 2 startx, y, z 145.868 600.485 291.922  wireno, peakt tpcno 403 2808.67 2 dist 320.054min T, max_T575.312 2826
no of hits closeby  0   event 10676 TrkackID 2 startx, y, z 145.868 600.485 291.922  wireno, peakt tpcno 403 2808.67 2 dist 320.054min T, max_T575.312 2826
27-Sep-2024 10:19:56 PDT  Closed output file "np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620_reco_stage1_reco_stage2_20240927T171956_keepup.root"
27-Sep-2024 10:19:56 PDT  Closed input file "np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              92.4495       153.493       338.225       144.686       48.0102        35     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000461293    0.0017929    0.00429068    0.00139982    0.00115025       35     
produce:fembfilter:PDHDFEMBFilter                      0.895595      0.912693      0.955926      0.913062      0.0100037       35     
produce:wclsdatahd:WireCellToolkit                      50.3843       52.8509       54.8547       53.0149       1.03539        35     
produce:gaushit:GausHitFinder                          0.739447       1.13917       1.93066       1.12809      0.244696        35     
produce:nhitsfilter:NumberOfHitsFilter                0.000233852   0.000341341   0.000579306   0.000320385   8.14238e-05      35     
produce:reco3d:SpacePointSolver                        0.256679       1.52196       6.68405       1.35796       1.13559        35     
produce:hitpdune:DisambigFromSpacePoints               0.328398       1.12205       2.55113       1.00995      0.516639        35     
produce:pandora:StandardPandora                         11.0722       42.6634       193.735       31.8555       35.1974        35     
produce:pandoraWriter:StandardPandora                  0.118933      0.196976      0.300956      0.195265      0.0409486       35     
produce:pandoraTrack:LArPandoraTrackCreation           0.854577       2.6323        6.04483       2.46446       1.15567        35     
produce:pandoraShower:LArPandoraShowerCreation         0.818643       2.82207       7.31003       2.59862       1.31748        35     
produce:pandoracalo:Calorimetry                        0.453936       1.28132       3.01475       1.23907      0.521726        35     
produce:pandoracalonosce:Calorimetry                   0.414442       1.20736       2.88546       1.15828      0.509232        35     
produce:pandorapid:Chi2ParticleID                     0.00165643    0.00262247    0.00405122    0.00251927    0.000565955      35     
produce:pandoraShowercalo:ShowerCalorimetry            0.753165       2.55923       7.24147       2.37456       1.23264        35     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.737647       2.4768        6.95867       2.29934       1.18654        35     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.100955      0.172367      0.229695      0.181535      0.0378983       35     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0361295     0.0592597     0.0835419     0.061593      0.0124707       35     
produce:emtrkmichelid:EmTrackMichelId                   22.5574       37.4659       54.1811       37.8957       7.51937        35     
[art]:TriggerResults:TriggerResultInserter            2.2536e-05    2.86508e-05   7.4934e-05    2.8256e-05    8.2915e-06       35     
end_path:out1:RootOutput                               9.185e-06    1.1267e-05    2.6421e-05    1.1002e-05    2.67262e-06      35     
end_path:michelremoving:michelremoving                 0.0205826     0.0730069     0.225546      0.0435542     0.0602158       35     
end_path:michelremovinggnocchi:michelremoving           0.0192       0.0698556     0.220551      0.0398016     0.0600071       35     
end_path:calibanagnocchi:CalibAnaTree                  0.312274       1.01927       2.44374      0.931771      0.456258        35     
end_path:out1:RootOutput(write)                        0.679267       1.16237       1.76757       1.17445       0.24535        35     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4993.34 MB
  Peak resident set size usage (VmHWM): 2670.56 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620_reco_stage1_reco_stage2_20240927T171956_keepup.root
\tHists: np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620_reco_stage1_reco_stage2_20240927T171956_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620_reco_stage1_reco_stage2_20240927T171956_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620_reco_stage1_reco_stage2_20240927T171956_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727452174.0,
    "core.end_time": 1727457596.0,
    "core.events": [
      10540,
      10544,
      10548,
      10552,
      10556,
      10560,
      10564,
      10568,
      10572,
      10576,
      10580,
      10584,
      10588,
      10592,
      10596,
      10600,
      10604,
      10608,
      10612,
      10616,
      10620,
      10624,
      10628,
      10632,
      10636,
      10640,
      10644,
      10648,
      10652,
      10656,
      10660,
      10664,
      10668,
      10672,
      10676
    ],
    "core.event_count": 35,
    "core.first_event_number": 10540,
    "core.last_event_number": 10676,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27410
    ],
    "core.runs_subruns": [
      2741000001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620_reco_stage1_reco_stage2_20240927T171956_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727452174.0,
    "core.end_time": 1727457596.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27410
    ],
    "core.runs_subruns": [
      2741000001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027410_0075_dataflow0_datawriter_0_20240625T021620.hdf5"
    }
  ]
}
justIN time: 2024-11-17 08:00:48 UTC       justIN version: 01.01.09