justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 269723.135@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID269723.135@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3592
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-10-05 02:16:16
SiteUS_BNL
EntryDUNE_US_BNL_sp02
Last heartbeat2024-10-05 05:54:46
From worker nodeHostnamespool1607.sdcc.bnl.gov
cpuinfoIntel(R) Xeon(R) Gold 6336Y CPU @ 2.40GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?False
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-10-05 04:28:27
Input fileshd-protodune:np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936.hdf5
Outputting started2024-10-05 05:52:56
Output files
Finished2024-10-05 05:54:46
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

+06 n=797911 tag=wiener
2452 hits in TPC 0
3312 hits in TPC 1
8582 hits in TPC 2
4133 hits in TPC 3
3412 hits in TPC 4
5431 hits in TPC 5
6022 hits in TPC 6
4718 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 5 XUs and 4 XVs -> 0 XUVs
C:0 T:1 4094 XUs and 2510 XVs -> 113 XUVs
C:0 T:2 105061 XUs and 71477 XVs -> 3746 XUVs
C:0 T:3 365 XUs and 445 XVs -> 5 XUVs
C:0 T:4 1004 XUs and 1168 XVs -> 25 XUVs
C:0 T:5 34464 XUs and 43930 XVs -> 1531 XUVs
C:0 T:6 62379 XUs and 88459 XVs -> 4795 XUVs
C:0 T:7 92 XUs and 56 XVs -> 1 XUVs
10216 XUVs total
1895 collection wire objects
10216 potential space points
Neighbour search...
1976266 tests to find 843874 neighbours
Iterating with no regularization...
Begin: 4.57643e+07
0 4.19883e+07
1 4.16702e+07
2 4.16181e+07
3 4.16028e+07
Now with regularization...
Begin: 3.95728e+07
0 3.95602e+07
TestBeamParticleCreationAlgorithm::SetupTestBeamVertex - Test beam particle has no initial vertex
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

05-Oct-2024 05:52:47 UTC  Closed output file "np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936_reco_stage1_reco_stage2_20241005T055247_keepup.root"
05-Oct-2024 05:52:47 UTC  Closed input file "np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              54.7157       114.533       159.842       118.38        24.6931        31     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00035549    0.00178753    0.00578613    0.00110298    0.00156849       31     
produce:fembfilter:PDHDFEMBFilter                      0.671785      0.706962      0.833255       0.69132      0.0374757       31     
produce:wclsdatahd:WireCellToolkit                      44.1705       48.2196       52.3263       48.0114       1.57404        31     
produce:gaushit:GausHitFinder                          0.422413       1.01887       3.55156      0.946573      0.497481        31     
produce:nhitsfilter:NumberOfHitsFilter                0.000311918   0.000635453   0.00246703     0.0005599    0.000358547      31     
produce:reco3d:SpacePointSolver                         0.23775       7.06741       14.9519       6.85979       3.71887        30     
produce:hitpdune:DisambigFromSpacePoints               0.145045      0.834947       1.5125       0.877702      0.318976        30     
produce:pandora:StandardPandora                         5.79345       23.3235       47.8813       21.3793       9.78713        30     
produce:pandoraWriter:StandardPandora                  0.0963898     0.210176      0.368402      0.211534      0.0524991       30     
produce:pandoraTrack:LArPandoraTrackCreation           0.637917       3.54806       7.9824        3.30518       1.61251        30     
produce:pandoraShower:LArPandoraShowerCreation         0.788557       4.2124        9.57749       4.34837       2.10422        30     
produce:pandoracalo:Calorimetry                        0.332344       1.39192       2.92276       1.40075      0.581608        30     
produce:pandoracalonosce:Calorimetry                   0.275592       1.31446       3.12379       1.33394      0.592948        30     
produce:pandorapid:Chi2ParticleID                     0.00121076    0.00220154     0.0038414    0.00214106    0.000554526      30     
produce:pandoraShowercalo:ShowerCalorimetry            0.550745       3.07833       5.55877       3.00206       1.40111        30     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.535245       3.1361        6.14989       3.43217       1.46024        30     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0576985      0.13737      0.206388      0.136098      0.0329105       30     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0280967     0.0614415     0.096614      0.0603469     0.0161452       30     
produce:emtrkmichelid:EmTrackMichelId                   7.87082       16.0851       23.0428       16.5554       3.20042        30     
[art]:TriggerResults:TriggerResultInserter            3.6064e-05    5.30756e-05   9.7334e-05    4.9575e-05    1.4897e-05       31     
end_path:out1:RootOutput                               7.52e-06     1.45421e-05   3.4314e-05    1.2583e-05    6.0993e-06       31     
end_path:michelremoving:michelremoving                 3.887e-05     0.0657484     0.200821      0.0463925     0.047971        31     
end_path:michelremovinggnocchi:michelremoving         1.7431e-05     0.0638785     0.225713      0.0428329     0.0529781       31     
end_path:calibanagnocchi:CalibAnaTree                  9.31e-06      0.929503       1.65924      0.916391      0.413957        31     
end_path:out1:RootOutput(write)                        0.390677      0.917326       1.27187      0.904936      0.208476        31     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5885.02 MB
  Peak resident set size usage (VmHWM): 2553.63 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936_reco_stage1_reco_stage2_20241005T055247_keepup.root
\tHists: np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936_reco_stage1_reco_stage2_20241005T055247_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936_reco_stage1_reco_stage2_20241005T055247_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936_reco_stage1_reco_stage2_20241005T055247_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1728103981.0,
    "core.end_time": 1728107568.0,
    "core.events": [
      95566,
      95574,
      95582,
      95590,
      95598,
      95606,
      95614,
      95622,
      95630,
      95638,
      95646,
      95654,
      95662,
      95670,
      95678,
      95686,
      95694,
      95702,
      95710,
      95718,
      95726,
      95734,
      95742,
      95750,
      95758,
      95766,
      95774,
      95782,
      95790,
      95798,
      95806
    ],
    "core.event_count": 31,
    "core.first_event_number": 95566,
    "core.last_event_number": 95806,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27882
    ],
    "core.runs_subruns": [
      2788200001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936_reco_stage1_reco_stage2_20241005T055247_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1728103981.0,
    "core.end_time": 1728107568.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      27882
    ],
    "core.runs_subruns": [
      2788200001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run027882_0385_dataflow2_datawriter_0_20240708T191936.hdf5"
    }
  ]
}
justIN time: 2024-11-21 20:14:00 UTC       justIN version: 01.01.09