justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 92090.26@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID92090.26@justin-prod-sched02.dune.hep.ac.uk
Workflow ID3934
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-11-06 14:32:31
SiteUK_RAL-Tier1
EntryLIGO_UK_RAL_arc_ce05
Last heartbeat2024-11-06 16:03:35
From worker nodeHostnamedune001-2668562.0-lcg2723.gridpp.rl.ac.uk
cpuinfoAMD EPYC 9654 96-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statestalled
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-11-06 14:33:33
Input fileshd-protodune:np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403.hdf5
Outputting started2024-11-06 15:59:35
Output files
Finished2024-11-06 16:49:40
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

l 10239
wclsFrameSaver: no samples within desired window for channel 10239
wclsFrameSaver: no samples within desired window for channel 10239
wclsFrameSaver: no samples within desired window for channel 10239
wclsFrameSaver: no samples within desired window for channel 10239
wclsFrameSaver: no samples within desired window for channel 10239
FrameSaver: q=1.83135e+06 n=1141130 tag=wiener
3055 hits in TPC 0
3394 hits in TPC 1
5779 hits in TPC 2
3904 hits in TPC 3
1811 hits in TPC 4
4142 hits in TPC 5
17711 hits in TPC 6
9186 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 732 XUs and 635 XVs -> 10 XUVs
C:0 T:2 12261 XUs and 14604 XVs -> 598 XUVs
C:0 T:3 51 XUs and 122 XVs -> 7 XUVs
C:0 T:4 36 XUs and 36 XVs -> 3 XUVs
C:0 T:5 6835 XUs and 14509 XVs -> 219 XUVs
C:0 T:6 187952 XUs and 250953 XVs -> 19369 XUVs
C:0 T:7 635 XUs and 1101 XVs -> 25 XUVs
20231 XUVs total
4367 collection wire objects
20231 potential space points
Neighbour search...
3723337 tests to find 878194 neighbours
Iterating with no regularization...
Begin: 3.63755e+07
0 2.28097e+07
1 2.14454e+07
2 2.12359e+07
3 2.11743e+07
4 2.11496e+07
5 2.11377e+07
Now with regularization...
Begin: 1.925e+07
0 1.9209e+07
1 1.91958e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

06-Nov-2024 15:59:21 UTC  Closed output file "np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403_reco_stage1_reco_stage2_20241106T155921_keepup.root"
06-Nov-2024 15:59:21 UTC  Closed input file "np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              75.5169       196.778       510.151       172.087       96.1298        20     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000490699   0.00439062     0.0135578    0.00232632    0.00399821       20     
produce:fembfilter:PDHDFEMBFilter                      0.872296       1.17139       1.8696        1.18958      0.216357        20     
produce:wclsdatahd:WireCellToolkit                      72.3943       94.0776       112.426       96.6339       11.6818        20     
produce:gaushit:GausHitFinder                          0.567319      0.971815       1.40246      0.934123       0.22469        20     
produce:nhitsfilter:NumberOfHitsFilter                0.000220803   0.000366453   0.000768096   0.000346782   0.000106881      20     
[art]:TriggerResults:TriggerResultInserter            3.0145e-05    7.32928e-05    0.0001192    7.7972e-05    1.86147e-05      20     
end_path:out1:RootOutput                              1.1056e-05    2.42395e-05    4.615e-05    2.3215e-05    8.26619e-06      20     
end_path:michelremoving:michelremoving                 3.292e-05     0.0532644      0.20897      0.0392412     0.0523468       20     
end_path:michelremovinggnocchi:michelremoving         1.4201e-05     0.049314      0.214305      0.0349345     0.0531986       20     
end_path:calibanagnocchi:CalibAnaTree                 1.0205e-05     0.814324       1.66172      0.800364      0.544363        20     
end_path:out1:RootOutput(write)                        0.218671      0.874996       1.25852      0.967929      0.345975        20     
produce:reco3d:SpacePointSolver                         1.70116       9.92118       40.9331       7.1733        9.76582        16     
produce:hitpdune:DisambigFromSpacePoints                0.66152       1.73564       3.69007       1.55839      0.739559        16     
produce:pandora:StandardPandora                         11.248        72.1912       321.62        35.4384       75.1418        16     
produce:pandoraWriter:StandardPandora                  0.174434       0.24561      0.321894      0.242389      0.039973        16     
produce:pandoraTrack:LArPandoraTrackCreation            2.02404       4.17626       7.38212       4.17051       1.46187        16     
produce:pandoraShower:LArPandoraShowerCreation          1.77253       3.98189       8.10404       3.89721       1.61818        16     
produce:pandoracalo:Calorimetry                        0.822821       1.40877       2.88503       1.26921      0.484632        16     
produce:pandoracalonosce:Calorimetry                   0.773798       1.32609       2.86384       1.15652      0.502658        16     
produce:pandorapid:Chi2ParticleID                     0.00167905    0.00290629    0.00477295    0.00277647    0.00077379       16     
produce:pandoraShowercalo:ShowerCalorimetry             1.42266       2.84226       5.77779       2.55773       1.12824        16     
produce:pandoraShowercalonosce:ShowerCalorimetry        1.42472       2.72017       4.52432       2.56089      0.871785        16     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.127843       0.18117      0.239356      0.179389      0.0293787       16     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.047821      0.0650261     0.103931      0.062362      0.014009        16     
produce:emtrkmichelid:EmTrackMichelId                   17.0018       21.9968       27.6424       22.1134       3.37356        16     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 7163.6 MB
  Peak resident set size usage (VmHWM): 4366.31 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403_reco_stage1_reco_stage2_20241106T155921_keepup.root
\tHists: np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403_reco_stage1_reco_stage2_20241106T155921_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403_reco_stage1_reco_stage2_20241106T155921_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403_reco_stage1_reco_stage2_20241106T155921_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1730904783.0,
    "core.end_time": 1730908762.0,
    "core.events": [
      44448,
      44450,
      44452,
      44454,
      44456,
      44458,
      44460,
      44462,
      44464,
      44466,
      44468,
      44470,
      44472,
      44474,
      44476,
      44478,
      44480,
      44482,
      44484,
      44486
    ],
    "core.event_count": 20,
    "core.first_event_number": 44448,
    "core.last_event_number": 44486,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      32672
    ],
    "core.runs_subruns": [
      3267200001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403_reco_stage1_reco_stage2_20241106T155921_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1730904783.0,
    "core.end_time": 1730908762.0,
    "core.data_stream": "cosmics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      32672
    ],
    "core.runs_subruns": [
      3267200001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run032672_1060_dataflow0_datawriter_0_20241106T115403.hdf5"
    }
  ]
}
justIN time: 2024-11-17 05:53:57 UTC       justIN version: 01.01.09