justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261562.166@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID261562.166@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3387
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-24 19:46:01
SiteUK_Liverpool
EntryDUNE_UK_Liverpool_hepgrid5
Last heartbeat2024-09-24 21:39:42
From worker nodeHostnamer12-n07.ph.liv.ac.uk
cpuinfoAMD EPYC 7453 28-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit259200 (72 hours)
Inner Apptainer?True
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-24 20:28:33
Input fileshd-protodune:np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907.hdf5
Outputting started2024-09-24 21:35:35
Output files
Finished2024-09-24 21:39:42
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

FrameSaver: q=1.1136e+06 n=815751 tag=wiener
1572 hits in TPC 0
1564 hits in TPC 1
7965 hits in TPC 2
4935 hits in TPC 3
2885 hits in TPC 4
6590 hits in TPC 5
6846 hits in TPC 6
3518 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 26 XUs and 15 XVs -> 0 XUVs
C:0 T:2 65980 XUs and 115271 XVs -> 4151 XUVs
C:0 T:3 1110 XUs and 1769 XVs -> 34 XUVs
C:0 T:4 187 XUs and 270 XVs -> 16 XUVs
C:0 T:5 16772 XUs and 13564 XVs -> 702 XUVs
C:0 T:6 39119 XUs and 59028 XVs -> 2558 XUVs
C:0 T:7 153 XUs and 192 XVs -> 8 XUVs
7469 XUVs total
2144 collection wire objects
7469 potential space points
Neighbour search...
792463 tests to find 374646 neighbours
Iterating with no regularization...
Begin: 8.57622e+06
0 7.90793e+06
1 7.85899e+06
2 7.85116e+06
Now with regularization...
Begin: 7.38697e+06
0 7.38354e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

24-Sep-2024 22:35:26 BST  Closed output file "np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907_reco_stage1_reco_stage2_20240924T213526_keepup.root"
24-Sep-2024 22:35:26 BST  Closed input file "np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              50.318        75.0067       114.927       74.3624       14.7636        30     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000385133   0.00253031     0.004124     0.00260511    0.000894122      30     
produce:fembfilter:PDHDFEMBFilter                      0.452623      0.496158      0.590413      0.488125      0.0266046       30     
produce:wclsdatahd:WireCellToolkit                      30.5291       31.5344       32.8853       31.4941      0.622793        30     
produce:gaushit:GausHitFinder                          0.270334      0.481011      0.674506       0.48296      0.114473        30     
produce:nhitsfilter:NumberOfHitsFilter                0.000196078   0.000304431   0.00047938    0.000297775   5.8773e-05       30     
produce:reco3d:SpacePointSolver                        0.145269      0.948431       3.42952       0.59944      0.835772        30     
produce:hitpdune:DisambigFromSpacePoints               0.110231      0.334622      0.671964      0.303739       0.16725        30     
produce:pandora:StandardPandora                         3.1992        9.18883       17.6368       8.49512       3.91627        30     
produce:pandoraWriter:StandardPandora                  0.0673842     0.118123      0.194676      0.119332      0.0305032       30     
produce:pandoraTrack:LArPandoraTrackCreation            0.2859        1.37491       3.54803       1.24621      0.731902        30     
produce:pandoraShower:LArPandoraShowerCreation         0.197673       1.5353        4.98069       1.47156       1.02334        30     
produce:pandoracalo:Calorimetry                        0.175697      0.615342       1.59935      0.561454      0.300807        30     
produce:pandoracalonosce:Calorimetry                   0.153899      0.584243       1.85413      0.521541      0.334623        30     
produce:pandorapid:Chi2ParticleID                     0.00092603    0.00159719    0.00261463    0.00154275    0.000376471      30     
produce:pandoraShowercalo:ShowerCalorimetry            0.199541       1.37166       4.58563       1.2021       0.922475        30     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.197641       1.27235       4.31571       1.12667      0.852747        30     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0432445     0.0944431     0.158173      0.0918137     0.0250612       30     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0169378     0.0352123     0.0634817     0.033922     0.00947356       30     
produce:emtrkmichelid:EmTrackMichelId                   13.3545       23.766        40.0115       23.2138       6.02646        30     
[art]:TriggerResults:TriggerResultInserter             2.123e-05    4.26556e-05   0.000107833   3.97695e-05   1.61177e-05      30     
end_path:out1:RootOutput                                5.3e-06     1.34974e-05   3.4986e-05    1.30045e-05   5.51381e-06      30     
end_path:michelremoving:michelremoving                 0.0125138     0.0588563     0.210518      0.0271138     0.0599401       30     
end_path:michelremovinggnocchi:michelremoving          0.0115593     0.0553785     0.208428      0.0253956     0.0585176       30     
end_path:calibanagnocchi:CalibAnaTree                  0.102239      0.418559       1.0841       0.367513      0.205178        30     
end_path:out1:RootOutput(write)                        0.290113       0.54626      0.843507      0.559549      0.128702        30     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4946.96 MB
  Peak resident set size usage (VmHWM): 2364.25 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907_reco_stage1_reco_stage2_20240924T213526_keepup.root
\tHists: np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907_reco_stage1_reco_stage2_20240924T213526_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907_reco_stage1_reco_stage2_20240924T213526_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907_reco_stage1_reco_stage2_20240924T213526_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727211437.0,
    "core.end_time": 1727213726.0,
    "core.events": [
      456008,
      456016,
      456024,
      456032,
      456040,
      456048,
      456056,
      456064,
      456072,
      456080,
      456088,
      456096,
      456104,
      456112,
      456120,
      456128,
      456136,
      456144,
      456152,
      456160,
      456168,
      456176,
      456184,
      456192,
      456200,
      456208,
      456216,
      456224,
      456232,
      456240
    ],
    "core.event_count": 30,
    "core.first_event_number": 456008,
    "core.last_event_number": 456240,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      29006
    ],
    "core.runs_subruns": [
      2900600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907_reco_stage1_reco_stage2_20240924T213526_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727211437.0,
    "core.end_time": 1727213726.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      29006
    ],
    "core.runs_subruns": [
      2900600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run029006_1900_dataflow0_datawriter_0_20240831T185907.hdf5"
    }
  ]
}
justIN time: 2024-09-29 09:26:39 UTC       justIN version: 01.01.08