Jobsub ID 263310.132@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263310.132@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 10:02:33 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 12:04:21 |
| From worker node | Hostname | wn006.hep |
| cpuinfo | Intel(R) Core(TM) i7-5960X CPU @ 3.00GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:21:07 |
| Input files | hd-protodune:np04hd_raw_run029092_0347_dataflow0_datawriter_0_20240905T223920.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 12:04:21 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
th regularization...
Begin: 7.56626e+06
0 7.56411e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 83520 TrkackID 13 startx, y, z 37.923 586.608 269.635 wireno, peakt tpcno 438 1619.02 5 dist 0.491356min T, max_T1611.67 2966.22
no of hits closeby 0 event 83520 TrkackID 13 startx, y, z 37.923 586.608 269.635 wireno, peakt tpcno 438 1619.02 5 dist 0.491356min T, max_T1611.67 2966.22
Begin processing the 29th record. run: 29092 subRun: 1 event: 83536 at 26-Sep-2024 12:41:58 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 44463 traces, 44463 summary
Retagger: tagging trace set: gauss with 33759 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 33759 traces tagged "gauss"
FrameSaver: q=754857 n=639059 tag=gauss
wclsFrameSaver: saving 44463 traces tagged "wiener"
FrameSaver: q=820027 n=582400 tag=wiener
1143 hits in TPC 0
1964 hits in TPC 1
4392 hits in TPC 2
2568 hits in TPC 3
1286 hits in TPC 4
5903 hits in TPC 5
4333 hits in TPC 6
3292 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 152 XUs and 58 XVs -> 0 XUVs
C:0 T:2 7932 XUs and 12580 XVs -> 571 XUVs
C:0 T:3 146 XUs and 162 XVs -> 4 XUVs
C:0 T:4 125 XUs and 342 XVs -> 10 XUVs
C:0 T:5 68166 XUs and 72535 XVs -> 3735 XUVs
C:0 T:6 7812 XUs and 9359 XVs -> 334 XUVs
C:0 T:7 216 XUs and 247 XVs -> 11 XUVs
4665 XUVs total
1298 collection wire objects
4665 potential space points
Neighbour search...
264511 tests to find 107092 neighbours
Iterating with no regularization...
Begin: 1.02333e+07
0 8.14629e+06
1 7.96519e+06
2 7.94164e+06
3 7.93493e+06
Now with regularization...
Begin: 7.52031e+06
0 7.51692e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29092 subRun: 1 event: 83544 at 26-Sep-2024 12:43:47 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 53452 traces, 53452 summary
Retagger: tagging trace set: gauss with 39345 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 39345 traces tagged "gauss"
FrameSaver: q=906714 n=719888 tag=gauss
wclsFrameSaver: saving 53452 traces tagged "wiener"
FrameSaver: q=973294 n=647672 tag=wiener
3558 hits in TPC 0
2803 hits in TPC 1
2941 hits in TPC 2
2587 hits in TPC 3
2401 hits in TPC 4
4506 hits in TPC 5
5699 hits in TPC 6
3521 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 880 XUs and 137 XVs -> 8 XUVs
C:0 T:2 17365 XUs and 16219 XVs -> 519 XUVs
C:0 T:3 37 XUs and 82 XVs -> 1 XUVs
C:0 T:4 32 XUs and 52 XVs -> 1 XUVs
C:0 T:5 22418 XUs and 11831 XVs -> 477 XUVs
C:0 T:6 18241 XUs and 17479 XVs -> 2136 XUVs
C:0 T:7 76 XUs and 66 XVs -> 9 XUVs
3151 XUVs total
840 collection wire objects
3151 potential space points
Neighbour search...
192277 tests to find 114644 neighbours
Iterating with no regularization...
Begin: 1.82024e+07
0 1.71202e+07
1 1.70393e+07
2 1.70223e+07
3 1.70164e+07
Now with regularization...
Begin: 1.60438e+07
0 1.60391e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 83544 TrkackID 0 startx, y, z 81.6958 301.435 8.98336 wireno, peakt tpcno 449 4113.99 5 dist 0min T, max_T3676.62 4726.79
no of hits closeby 1 event 83544 TrkackID 0 startx, y, z 81.6958 301.435 8.98336 wireno, peakt tpcno 449 4113.99 5 dist 0min T, max_T3676.62 4726.79
26-Sep-2024 12:45:30 BST Closed output file "np04hd_raw_run029092_0347_dataflow0_datawriter_0_20240905T223920_reco_stage1_reco_stage2_20240926T114530_keepup.root"
26-Sep-2024 12:45:30 BST Closed input file "np04hd_raw_run029092_0347_dataflow0_datawriter_0_20240905T223920_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 79.2023 114.699 135.944 114.513 13.2758 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000335644 0.0019047 0.0030419 0.00198195 0.00078472 30
produce:fembfilter:PDHDFEMBFilter 0.541847 0.79512 4.09987 0.639353 0.625761 30
produce:wclsdatahd:WireCellToolkit 53.8888 57.7586 67.8932 57.1175 2.92046 30
produce:gaushit:GausHitFinder 0.319265 0.712121 1.1983 0.686157 0.186649 30
produce:nhitsfilter:NumberOfHitsFilter 0.00018374 0.000348946 0.00052928 0.000335716 7.41641e-05 30
produce:reco3d:SpacePointSolver 0.111131 1.08278 4.29268 0.848065 0.90894 30
produce:hitpdune:DisambigFromSpacePoints 0.100642 0.640943 1.76202 0.541114 0.322381 30
produce:pandora:StandardPandora 3.07063 14.335 23.0852 13.9667 4.38364 30
produce:pandoraWriter:StandardPandora 0.0682721 0.146256 0.215259 0.142317 0.0327946 30
produce:pandoraTrack:LArPandoraTrackCreation 0.366913 1.40633 2.19139 1.30352 0.431772 30
produce:pandoraShower:LArPandoraShowerCreation 0.365205 1.66472 3.76374 1.65973 0.646479 30
produce:pandoracalo:Calorimetry 0.217411 0.746793 1.14018 0.731251 0.206896 30
produce:pandoracalonosce:Calorimetry 0.194 0.667349 1.02398 0.655846 0.187478 30
produce:pandorapid:Chi2ParticleID 0.000898085 0.00165318 0.00301506 0.00152726 0.000404221 30
produce:pandoraShowercalo:ShowerCalorimetry 0.328467 1.59968 3.28162 1.58254 0.583782 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.329074 1.54842 3.40348 1.52301 0.602323 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.06466 0.125021 0.188354 0.12149 0.0315854 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0220814 0.0456388 0.107403 0.0436378 0.0160265 30
produce:emtrkmichelid:EmTrackMichelId 16.1308 29.8374 39.1309 30.4099 5.13359 30
[art]:TriggerResults:TriggerResultInserter 1.8134e-05 3.40637e-05 6.4153e-05 3.3595e-05 1.0259e-05 30
end_path:out1:RootOutput 3.512e-06 6.04537e-06 2.0736e-05 4.9965e-06 3.38434e-06 30
end_path:michelremoving:michelremoving 0.0150686 0.115148 0.313665 0.0758128 0.0902235 30
end_path:michelremovinggnocchi:michelremoving 0.0153109 0.106625 0.263546 0.0740516 0.0825659 30
end_path:calibanagnocchi:CalibAnaTree 0.14351 0.571505 1.06814 0.559436 0.191133 30
end_path:out1:RootOutput(write) 0.397477 0.673947 0.94035 0.675627 0.126883 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4919.09 MB
Peak resident set size usage (VmHWM): 2336.33 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0347_dataflow0_datawriter_0_20240905T223920_reco_stage1_reco_stage2_20240926T114530_keepup.root
\tHists: np04hd_raw_run029092_0347_dataflow0_datawriter_0_20240905T223920_reco_stage1_reco_stage2_20240926T114530_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
