justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 261562.148@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID261562.148@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3387
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-24 19:46:01
SiteCERN
EntryCMSHTPC_T2_CH_CERN_ce504
Last heartbeat2024-09-24 21:39:57
From worker nodeHostnameb9p12p3085.cern.ch
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job stateaborted
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-24 20:28:18
Input fileshd-protodune:np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714.hdf5
Outputting started2024-09-24 21:38:39
Output files
Finished2024-09-24 21:39:57
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

race set: gauss with 47219 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47219 traces tagged "gauss"
FrameSaver: q=1.0167e+06 n=909115 tag=gauss
wclsFrameSaver: saving 62872 traces tagged "wiener"
FrameSaver: q=1.08155e+06 n=821750 tag=wiener
4179 hits in TPC 0
2099 hits in TPC 1
6929 hits in TPC 2
5043 hits in TPC 3
3773 hits in TPC 4
5464 hits in TPC 5
4032 hits in TPC 6
2991 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 5 XUs and 3 XVs -> 0 XUVs
C:0 T:2 42673 XUs and 36402 XVs -> 1425 XUVs
C:0 T:3 10 XUs and 8 XVs -> 1 XUVs
C:0 T:4 239 XUs and 262 XVs -> 19 XUVs
C:0 T:5 12772 XUs and 15647 XVs -> 415 XUVs
C:0 T:6 14930 XUs and 7233 XVs -> 241 XUVs
C:0 T:7 414 XUs and 552 XVs -> 25 XUVs
2126 XUVs total
993 collection wire objects
2126 potential space points
Neighbour search...
69360 tests to find 45434 neighbours
Iterating with no regularization...
Begin: 5.67342e+06
0 5.12115e+06
1 5.07934e+06
2 5.0735e+06
3 5.07158e+06
Now with regularization...
Begin: 4.75873e+06
0 4.75751e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

24-Sep-2024 23:38:23 CEST  Closed output file "np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714_reco_stage1_reco_stage2_20240924T213823_keepup.root"
24-Sep-2024 23:38:23 CEST  Closed input file "np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              75.7066       104.25        137.732       107.588       15.1865        30     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000786385    0.0026576    0.00424379    0.00273535    0.000856415      30     
produce:fembfilter:PDHDFEMBFilter                      0.564748      0.744931      0.868453      0.770867      0.0819466       30     
produce:wclsdatahd:WireCellToolkit                      37.0229       42.8132       46.6229       43.3397       2.11454        30     
produce:gaushit:GausHitFinder                          0.440112      0.772937       1.16736      0.758032      0.164125        30     
produce:nhitsfilter:NumberOfHitsFilter                0.000224281   0.000346921   0.000566263   0.000343512   6.51487e-05      30     
produce:reco3d:SpacePointSolver                        0.242237       1.13295       4.24191      0.854826       0.82562        30     
produce:hitpdune:DisambigFromSpacePoints               0.284129      0.663525       1.60171      0.593768      0.312358        30     
produce:pandora:StandardPandora                         6.83767       13.5122       21.1884       13.4219       3.96427        30     
produce:pandoraWriter:StandardPandora                  0.0739077     0.162686       0.24835      0.167966      0.0425808       30     
produce:pandoraTrack:LArPandoraTrackCreation            0.47784       1.2116        2.38379       1.11372      0.480427        30     
produce:pandoraShower:LArPandoraShowerCreation          0.60959        1.419        3.14364       1.3218       0.632621        30     
produce:pandoracalo:Calorimetry                         0.29184      0.672086       1.26653      0.633632      0.239888        30     
produce:pandoracalonosce:Calorimetry                    0.23456      0.606202       1.14327      0.574857      0.217065        30     
produce:pandorapid:Chi2ParticleID                     0.00101261    0.00201355    0.00389732    0.00198144    0.000627276      30     
produce:pandoraShowercalo:ShowerCalorimetry            0.507187       1.34045       3.20423       1.27375      0.592427        30     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.471752       1.3206        3.26202       1.29491       0.58923        30     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0772019     0.157475      0.249448      0.153518      0.0368264       30     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0363812     0.054282      0.0803934     0.0536413     0.0122427       30     
produce:emtrkmichelid:EmTrackMichelId                   23.4135       35.9533       52.5439       36.8035       7.31026        30     
[art]:TriggerResults:TriggerResultInserter             2.485e-05    3.76708e-05   9.7261e-05    3.3285e-05    1.35919e-05      30     
end_path:out1:RootOutput                               8.68e-06     1.21658e-05    3.471e-05    1.0785e-05    5.39219e-06      30     
end_path:michelremoving:michelremoving                 0.0173817     0.106469      0.385982      0.0421025     0.102231        30     
end_path:michelremovinggnocchi:michelremoving          0.0157213      0.10148      0.374986      0.0349799     0.100003        30     
end_path:calibanagnocchi:CalibAnaTree                  0.216399      0.485582       0.94869       0.45046      0.173524        30     
end_path:out1:RootOutput(write)                        0.605243      0.883515       1.35752      0.889354      0.179003        30     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4926.56 MB
  Peak resident set size usage (VmHWM): 2587.4 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714_reco_stage1_reco_stage2_20240924T213823_keepup.root
\tHists: np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714_reco_stage1_reco_stage2_20240924T213823_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714_reco_stage1_reco_stage2_20240924T213823_keepup.root
Ran successfully
{
  "name": "np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714_reco_stage1_reco_stage2_20240924T213823_keepup.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "artroot",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "full-reconstructed",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727210729.0,
    "core.end_time": 1727213904.0,
    "core.events": [
      455763,
      455771,
      455779,
      455787,
      455795,
      455803,
      455811,
      455819,
      455827,
      455835,
      455843,
      455851,
      455859,
      455867,
      455875,
      455883,
      455891,
      455899,
      455907,
      455915,
      455923,
      455931,
      455939,
      455947,
      455955,
      455963,
      455971,
      455979,
      455987,
      455995
    ],
    "core.event_count": 30,
    "core.first_event_number": 455763,
    "core.last_event_number": 455995,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      29006
    ],
    "core.runs_subruns": [
      2900600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714.hdf5"
    }
  ]
}Forming hist metadata
formed
{
  "name": "np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714_reco_stage1_reco_stage2_20240924T213823_keepup_hists.root",
  "namespace": "hd-protodune-det-reco",
  "metadata": {
    "core.file_format": "root",
    "core.application.name": "reco",
    "core.application.family": "dunesw",
    "core.application.version": "v09_91_02d01",
    "core.data_tier": "root-tuple-virtual",
    "dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
    "dune.campaign": "hd-protodune-reco-keepup-v0",
    "core.start_time": 1727210729.0,
    "core.end_time": 1727213904.0,
    "core.data_stream": "physics",
    "core.file_content_status": "good",
    "core.file_type": "detector",
    "core.run_type": "hd-protodune",
    "core.runs": [
      29006
    ],
    "core.runs_subruns": [
      2900600001
    ],
    "dune.daq_test": false,
    "retention.status": "active",
    "retention.class": "physics",
    "origin.applications.config_files": {
      "reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
    },
    "origin.applications.versions": {
      "reco1": "v09_91_02d01"
    },
    "origin.applications.names": [
      "reco1"
    ]
  },
  "parents": [
    {
      "did": "hd-protodune:np04hd_raw_run029006_1899_dataflow3_datawriter_0_20240831T185714.hdf5"
    }
  ]
}
justIN time: 2024-11-17 06:09:39 UTC       justIN version: 01.01.09