justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 276383.127@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID276383.127@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3705
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-10-17 11:32:12
SiteUS_BNL
EntryDUNE_US_BNL_sp02
Last heartbeat2024-10-17 13:26:22
From worker nodeHostnamespool1606.sdcc.bnl.gov
cpuinfoIntel(R) Xeon(R) Gold 6336Y CPU @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?False
Job statestalled
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-10-17 11:36:12
Input fileshd-protodune:np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258.hdf5
Outputting started2024-10-17 13:22:30
Output files
Finished2024-10-17 14:25:27
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ticks
wclsFrameSaver: saving 61302 traces tagged "gauss"
FrameSaver: q=2.39542e+06 n=1093866 tag=gauss
wclsFrameSaver: saving 80800 traces tagged "wiener"
FrameSaver: q=2.59263e+06 n=987810 tag=wiener
4397 hits in TPC 0
4316 hits in TPC 1
6018 hits in TPC 2
3623 hits in TPC 3
7643 hits in TPC 4
11108 hits in TPC 5
4148 hits in TPC 6
2288 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 3 XUs and 0 XVs -> 0 XUVs
C:0 T:1 9119 XUs and 2777 XVs -> 60 XUVs
C:0 T:2 71867 XUs and 39030 XVs -> 3247 XUVs
C:0 T:3 90 XUs and 135 XVs -> 4 XUVs
C:0 T:4 447 XUs and 469 XVs -> 20 XUVs
C:0 T:5 112374 XUs and 66540 XVs -> 4603 XUVs
C:0 T:6 10751 XUs and 7705 XVs -> 205 XUVs
C:0 T:7 500 XUs and 304 XVs -> 10 XUVs
8149 XUVs total
2095 collection wire objects
8149 potential space points
Neighbour search...
770493 tests to find 396894 neighbours
Iterating with no regularization...
Begin: 8.63426e+07
0 7.37615e+07
1 7.3126e+07
2 7.3018e+07
3 7.29849e+07
Now with regularization...
Begin: 7.02304e+07
0 7.02107e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

17-Oct-2024 13:22:17 UTC  Closed output file "np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258_reco_stage1_reco_stage2_20241017T132217_keepup.root"
17-Oct-2024 13:22:17 UTC  Closed input file "np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              84.6361       114.242       174.339       109.229       21.3019        35     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000351177   0.00254765    0.00578145    0.00234387     0.0015453       35     
produce:fembfilter:PDHDFEMBFilter                      0.667011      0.814748       1.05228       0.77593      0.103821        35     
produce:wclsdatahd:WireCellToolkit                      51.4973       59.3547       71.8808       57.8718       5.00992        35     
produce:gaushit:GausHitFinder                          0.419396      0.945388       1.33619      0.942428      0.219063        35     
produce:nhitsfilter:NumberOfHitsFilter                0.000448527   0.000674091   0.00196145    0.00063513    0.000258755      35     
produce:reco3d:SpacePointSolver                        0.211706       1.11118       5.00087      0.841163      0.911221        35     
produce:hitpdune:DisambigFromSpacePoints               0.124275      0.515415       1.29104      0.452565      0.233978        35     
produce:pandora:StandardPandora                         5.22241       15.0131       41.036        12.119        7.49771        35     
produce:pandoraWriter:StandardPandora                  0.0985424     0.219126      0.471183      0.199104      0.0847612       35     
produce:pandoraTrack:LArPandoraTrackCreation            0.89193       2.98468       7.85641       2.66901       1.52948        35     
produce:pandoraShower:LArPandoraShowerCreation         0.970154       3.31384       7.53482       3.01241       1.67265        35     
produce:pandoracalo:Calorimetry                        0.403018       1.12273       2.25643       1.07712      0.476478        35     
produce:pandoracalonosce:Calorimetry                   0.441261       1.08053       2.56356      0.986738      0.453806        35     
produce:pandorapid:Chi2ParticleID                     0.00137676    0.00250449     0.0045098    0.00236618    0.000693373      35     
produce:pandoraShowercalo:ShowerCalorimetry            0.706765       2.67899       6.0187        2.42292       1.32447        35     
produce:pandoraShowercalonosce:ShowerCalorimetry        0.66925       2.63988       5.85146       2.44172       1.2869         35     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0547698     0.131994      0.264508      0.126839      0.0436632       35     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0254681     0.0573309     0.127711      0.054331      0.021234        35     
produce:emtrkmichelid:EmTrackMichelId                   11.8764       20.0461       30.9338       19.2794       4.0413         35     
[art]:TriggerResults:TriggerResultInserter            3.7255e-05    8.21694e-05   0.000192784   7.4514e-05    3.46108e-05      35     
end_path:out1:RootOutput                              1.2369e-05    2.23843e-05   4.0934e-05    2.0448e-05    7.22198e-06      35     
end_path:michelremoving:michelremoving                 0.0245436     0.120083      0.466235      0.0835854     0.0995363       35     
end_path:michelremovinggnocchi:michelremoving          0.0267565     0.107397      0.289088      0.0785574     0.0787781       35     
end_path:calibanagnocchi:CalibAnaTree                  0.389272      0.840462       1.63836      0.808578      0.315443        35     
end_path:out1:RootOutput(write)                        0.504991      0.810252       1.38375      0.767986       0.18945        35     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5832.02 MB
  Peak resident set size usage (VmHWM): 2522.99 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258_reco_stage1_reco_stage2_20241017T132217_keepup.root
\tHists: np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258_reco_stage1_reco_stage2_20241017T132217_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258_reco_stage1_reco_stage2_20241017T132217_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258_reco_stage1_reco_stage2_20241017T132217_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1729167292.0,
    "core.end_time": 1729171338.0,
    "core.events": [
      33604,
      33608,
      33612,
      33616,
      33620,
      33624,
      33628,
      33632,
      33636,
      33640,
      33644,
      33648,
      33652,
      33656,
      33660,
      33664,
      33668,
      33672,
      33676,
      33680,
      33684,
      33688,
      33692,
      33696,
      33700,
      33704,
      33708,
      33712,
      33716,
      33720,
      33724,
      33728,
      33732,
      33736,
      33740
    ],
    "core.event_count": 35,
    "core.first_event_number": 33604,
    "core.last_event_number": 33740,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27406
    ],
    "core.runs_subruns": [
      2740600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258_reco_stage1_reco_stage2_20241017T132217_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1729167292.0,
    "core.end_time": 1729171338.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27406
    ],
    "core.runs_subruns": [
      2740600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027406_0240_dataflow0_datawriter_0_20240624T145258.hdf5"
    }
  ]
}
justIN time: 2024-11-24 10:12:30 UTC       justIN version: 01.01.09