Jobsub ID 79974.181@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 79974.181@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3511 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-27 08:28:00 |
| Site | US_UCSD |
| Entry | CMSHTPC_T2_US_UCSD_gw7 |
| Last heartbeat | 2024-09-27 16:53:06 |
| From worker node | Hostname | mh-7763-8.t2.ucsd.edu |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-27 13:49:43 |
| Input files | hd-protodune:np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215.hdf5
|
| Jobscript | Exit code | 0 |
| Real time | 2h (9101s) |
| CPU time | 1h (3677s = 40%) |
| Outputting started | 2024-09-27 16:21:26 |
| Output files | |
| Finished | 2024-09-27 16:53:06 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
47876e+06 n=1157668 tag=gauss
wclsFrameSaver: saving 87789 traces tagged "wiener"
FrameSaver: q=2.72223e+06 n=1053842 tag=wiener
4147 hits in TPC 0
3994 hits in TPC 1
9451 hits in TPC 2
5529 hits in TPC 3
6852 hits in TPC 4
8229 hits in TPC 5
6717 hits in TPC 6
4749 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 5 XUs and 5 XVs -> 0 XUVs
C:0 T:1 23548 XUs and 14881 XVs -> 1784 XUVs
C:0 T:2 60951 XUs and 92600 XVs -> 3394 XUVs
C:0 T:3 61 XUs and 125 XVs -> 0 XUVs
C:0 T:4 894 XUs and 724 XVs -> 29 XUVs
C:0 T:5 60750 XUs and 38904 XVs -> 2845 XUVs
C:0 T:6 22615 XUs and 38886 XVs -> 1873 XUVs
C:0 T:7 1297 XUs and 814 XVs -> 37 XUVs
9962 XUVs total
2850 collection wire objects
9962 potential space points
Neighbour search...
823466 tests to find 393352 neighbours
Iterating with no regularization...
Begin: 7.82234e+07
0 6.93884e+07
1 6.89206e+07
2 6.88661e+07
Now with regularization...
Begin: 6.59658e+07
0 6.59443e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
27-Sep-2024 09:21:19 PDT Closed output file "np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215_reco_stage1_reco_stage2_20240927T162119_keepup.root"
27-Sep-2024 09:21:19 PDT Closed input file "np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 0.485128 88.4489 148.565 88.6375 39.8132 35
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000282786 0.00141322 0.00389748 0.000983319 0.000985963 35
produce:fembfilter:PDHDFEMBFilter 0.473057 0.576527 0.675212 0.577116 0.0389197 35
produce:wclsdatahd:WireCellToolkit 25.5016 28.6665 33.6997 28.3106 2.4514 31
produce:gaushit:GausHitFinder 0.418235 0.643019 1.08314 0.617615 0.131641 31
produce:nhitsfilter:NumberOfHitsFilter 0.000189498 0.000298832 0.000478835 0.000283466 5.49523e-05 31
produce:reco3d:SpacePointSolver 0.353719 1.22982 3.70846 1.02777 0.739234 31
produce:hitpdune:DisambigFromSpacePoints 0.249189 0.643899 1.44206 0.579333 0.283507 31
produce:pandora:StandardPandora 7.39896 28.6919 59.5807 25.1169 14.0088 31
produce:pandoraWriter:StandardPandora 0.073069 0.131516 0.195485 0.126529 0.032298 31
produce:pandoraTrack:LArPandoraTrackCreation 0.747964 1.78627 3.31071 1.73389 0.615472 31
produce:pandoraShower:LArPandoraShowerCreation 0.79789 1.81198 4.76367 1.51979 0.909676 31
produce:pandoracalo:Calorimetry 0.420749 0.959421 1.75627 0.898426 0.326351 31
produce:pandoracalonosce:Calorimetry 0.395307 0.91321 1.66675 0.846875 0.312561 31
produce:pandorapid:Chi2ParticleID 0.00136238 0.00193585 0.00346764 0.00192014 0.000397808 31
produce:pandoraShowercalo:ShowerCalorimetry 0.839027 1.83755 4.95605 1.52909 0.960561 31
produce:pandoraShowercalonosce:ShowerCalorimetry 0.794351 1.74602 4.88624 1.45995 0.925426 31
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0678888 0.100942 0.177993 0.0935999 0.0235816 31
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0245021 0.0355166 0.0656902 0.0333132 0.00924135 31
produce:emtrkmichelid:EmTrackMichelId 17.0533 28.3584 45.3816 26.2778 7.3268 31
[art]:TriggerResults:TriggerResultInserter 1.625e-05 2.78195e-05 8.3969e-05 2.312e-05 1.2346e-05 35
end_path:out1:RootOutput 3.85e-06 7.2294e-06 2.8009e-05 6.111e-06 4.19835e-06 35
end_path:michelremoving:michelremoving 1.566e-05 0.0376698 0.17007 0.0235668 0.0390364 35
end_path:michelremovinggnocchi:michelremoving 4.64e-06 0.033817 0.165097 0.0196499 0.0383076 35
end_path:calibanagnocchi:CalibAnaTree 4.1e-06 0.639811 1.29606 0.662063 0.332495 35
end_path:out1:RootOutput(write) 0.000440205 0.652172 1.20197 0.704762 0.274964 35
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5106.86 MB
Peak resident set size usage (VmHWM): 2717.97 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215_reco_stage1_reco_stage2_20240927T162119_keepup.root
\tHists: np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215_reco_stage1_reco_stage2_20240927T162119_keepup_hists.root
Forming reco metadata
Successfully opened file np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215_reco_stage1_reco_stage2_20240927T162119_keepup.root
Ran successfully
{
"name": "np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215_reco_stage1_reco_stage2_20240927T162119_keepup.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "artroot",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "full-reconstructed",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727450955.0,
"core.end_time": 1727454079.0,
"core.events": [
15683,
15687,
15691,
15695,
15699,
15703,
15707,
15711,
15715,
15719,
15723,
15727,
15731,
15735,
15739,
15743,
15747,
15751,
15755,
15759,
15763,
15767,
15771,
15775,
15779,
15783,
15787,
15791,
15795,
15799,
15803,
15807,
15811,
15815,
15819
],
"core.event_count": 35,
"core.first_event_number": 15683,
"core.last_event_number": 15819,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27334
],
"core.runs_subruns": [
2733400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215.hdf5"
}
]
}Forming hist metadata
formed
{
"name": "np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215_reco_stage1_reco_stage2_20240927T162119_keepup_hists.root",
"namespace": "hd-protodune-det-reco",
"metadata": {
"core.file_format": "root",
"core.application.name": "reco",
"core.application.family": "dunesw",
"core.application.version": "v09_91_02d01",
"core.data_tier": "root-tuple-virtual",
"dune.config_file": "standard_reco_stage2_calibration_protodunehd_keepup.fcl",
"dune.campaign": "hd-protodune-reco-keepup-v0",
"core.start_time": 1727450955.0,
"core.end_time": 1727454079.0,
"core.data_stream": "physics",
"core.file_content_status": "good",
"core.file_type": "detector",
"core.run_type": "hd-protodune",
"core.runs": [
27334
],
"core.runs_subruns": [
2733400001
],
"dune.daq_test": false,
"retention.status": "active",
"retention.class": "physics",
"origin.applications.config_files": {
"reco1": "standard_reco_stage1_protodunehd_keepup.fcl"
},
"origin.applications.versions": {
"reco1": "v09_91_02d01"
},
"origin.applications.names": [
"reco1"
]
},
"parents": [
{
"did": "hd-protodune:np04hd_raw_run027334_0112_dataflow3_datawriter_0_20240621T041215.hdf5"
}
]
} 