Jobsub ID 78721.186@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78721.186@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 11:42:40 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce02 |
| Last heartbeat | 2024-09-26 14:16:21 |
| From worker node | Hostname | wn5917301.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7702P 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:39:30 |
| Input files | hd-protodune:np04hd_raw_run029081_0675_dataflow1_datawriter_0_20240905T034718.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:16:21 |
| Saved logs | justin-logs:78721.186-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
-------------------------------------------------------------------------------------
Begin processing the 28th record. run: 29081 subRun: 1 event: 162241 at 26-Sep-2024 15:09:19 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 85387 traces, 85387 summary
Retagger: tagging trace set: gauss with 63236 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 63236 traces tagged "gauss"
FrameSaver: q=1.3854e+06 n=1134486 tag=gauss
wclsFrameSaver: saving 85387 traces tagged "wiener"
FrameSaver: q=1.49947e+06 n=1027542 tag=wiener
3471 hits in TPC 0
3534 hits in TPC 1
6589 hits in TPC 2
3848 hits in TPC 3
7791 hits in TPC 4
11219 hits in TPC 5
4644 hits in TPC 6
2560 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3728 XUs and 905 XVs -> 36 XUVs
C:0 T:2 69350 XUs and 60399 XVs -> 3276 XUVs
C:0 T:3 1342 XUs and 1153 XVs -> 26 XUVs
C:0 T:4 660 XUs and 655 XVs -> 54 XUVs
C:0 T:5 59342 XUs and 50550 XVs -> 1037 XUVs
C:0 T:6 29494 XUs and 29475 XVs -> 951 XUVs
C:0 T:7 798 XUs and 609 XVs -> 29 XUVs
5409 XUVs total
1720 collection wire objects
5409 potential space points
Neighbour search...
233059 tests to find 132280 neighbours
Iterating with no regularization...
Begin: 9.7488e+06
0 8.71411e+06
1 8.62207e+06
2 8.60785e+06
3 8.60337e+06
Now with regularization...
Begin: 8.04292e+06
0 8.04058e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 162249 at 26-Sep-2024 15:12:01 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 85245 traces, 85245 summary
Retagger: tagging trace set: gauss with 62060 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 62060 traces tagged "gauss"
FrameSaver: q=1.3161e+06 n=1120377 tag=gauss
wclsFrameSaver: saving 85245 traces tagged "wiener"
FrameSaver: q=1.44313e+06 n=1014784 tag=wiener
6802 hits in TPC 0
7926 hits in TPC 1
5746 hits in TPC 2
2171 hits in TPC 3
3600 hits in TPC 4
5416 hits in TPC 5
7706 hits in TPC 6
4734 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 17 XUs and 6 XVs -> 0 XUVs
C:0 T:1 21066 XUs and 15774 XVs -> 634 XUVs
C:0 T:2 25889 XUs and 34334 XVs -> 740 XUVs
C:0 T:3 4 XUs and 12 XVs -> 2 XUVs
C:0 T:4 251 XUs and 219 XVs -> 5 XUVs
C:0 T:5 49795 XUs and 33498 XVs -> 1061 XUVs
C:0 T:6 83617 XUs and 61185 XVs -> 3454 XUVs
C:0 T:7 505 XUs and 404 XVs -> 34 XUVs
5930 XUVs total
1661 collection wire objects
5930 potential space points
Neighbour search...
462992 tests to find 229076 neighbours
Iterating with no regularization...
Begin: 9.66071e+06
0 8.81023e+06
1 8.74821e+06
2 8.74285e+06
Now with regularization...
Begin: 8.37841e+06
0 8.37636e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 162249 TrkackID 3 startx, y, z 92.414 600.436 55.1263 wireno, peakt tpcno 473 1919.84 5 dist 0min T, max_T1919.84 3786.27
no of hits closeby 0 event 162249 TrkackID 3 startx, y, z 92.414 600.436 55.1263 wireno, peakt tpcno 473 1919.84 5 dist 0min T, max_T1919.84 3786.27
Begin processing the 30th record. run: 29081 subRun: 1 event: 162257 at 26-Sep-2024 15:14:36 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 240919 traces, 240919 summary
Retagger: tagging trace set: gauss with 193378 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 193378 traces tagged "gauss"
FrameSaver: q=7.17124e+06 n=3837868 tag=gauss
wclsFrameSaver: saving 240919 traces tagged "wiener"
FrameSaver: q=8.0001e+06 n=3528507 tag=wiener
10438 hits in TPC 0
19940 hits in TPC 1
26-Sep-2024 15:15:47 BST Closed output file "np04hd_raw_run029081_0675_dataflow1_datawriter_0_20240905T034718_reco_stage1_reco_stage2_20240926T141547_keepup.root"
26-Sep-2024 15:15:47 BST Closed input file "np04hd_raw_run029081_0675_dataflow1_datawriter_0_20240905T034718_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 70.0491 129.742 177.606 128.633 26.4717 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000655236 0.0031887 0.00534636 0.00357111 0.00125703 30
produce:fembfilter:PDHDFEMBFilter 0.712738 0.94848 1.04327 0.975251 0.0881601 30
produce:wclsdatahd:WireCellToolkit 40.5524 51.4899 61.5349 51.5302 3.17964 30
produce:gaushit:GausHitFinder 0.611132 1.25105 5.68736 1.12382 0.866154 30
produce:nhitsfilter:NumberOfHitsFilter 0.000332197 0.000508758 0.00155057 0.000470402 0.000213885 30
produce:reco3d:SpacePointSolver 0.449533 2.0049 5.54569 1.5638 1.31256 29
produce:hitpdune:DisambigFromSpacePoints 0.27531 0.870601 1.67909 0.833993 0.365345 29
produce:pandora:StandardPandora 6.36254 22.1133 48.6719 20.4171 10.3098 29
produce:pandoraWriter:StandardPandora 0.122142 0.273444 0.411079 0.269941 0.0732474 29
produce:pandoraTrack:LArPandoraTrackCreation 0.737226 1.96059 3.7885 1.9051 0.766023 29
produce:pandoraShower:LArPandoraShowerCreation 0.621618 2.26753 5.26304 2.27473 1.08745 29
produce:pandoracalo:Calorimetry 0.372729 0.957535 1.68326 0.932041 0.336685 29
produce:pandoracalonosce:Calorimetry 0.336043 0.863286 1.63342 0.834452 0.311094 29
produce:pandorapid:Chi2ParticleID 0.00163912 0.00282512 0.00498773 0.00291729 0.000735459 29
produce:pandoraShowercalo:ShowerCalorimetry 0.534461 2.03075 4.44877 1.95067 0.942532 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.505966 1.96566 4.29572 1.90151 0.961572 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.108011 0.225553 0.347581 0.223019 0.0562978 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0367399 0.0759275 0.114081 0.0761707 0.0190709 29
produce:emtrkmichelid:EmTrackMichelId 22.5766 40.5177 54.4437 40.9466 8.80088 29
[art]:TriggerResults:TriggerResultInserter 1.9948e-05 5.10961e-05 9.4498e-05 5.2128e-05 1.9261e-05 30
end_path:out1:RootOutput 4.419e-06 9.30357e-06 3.749e-05 6.467e-06 6.98975e-06 30
end_path:michelremoving:michelremoving 3.4806e-05 0.124016 0.463897 0.0498466 0.134769 30
end_path:michelremovinggnocchi:michelremoving 2.2643e-05 0.123227 0.468018 0.0497661 0.134552 30
end_path:calibanagnocchi:CalibAnaTree 1.3816e-05 0.728463 1.4128 0.718769 0.32789 30
end_path:out1:RootOutput(write) 0.705336 1.2973 1.77069 1.33742 0.296412 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4940 MB
Peak resident set size usage (VmHWM): 2606.29 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0675_dataflow1_datawriter_0_20240905T034718_reco_stage1_reco_stage2_20240926T141547_keepup.root
\tHists: np04hd_raw_run029081_0675_dataflow1_datawriter_0_20240905T034718_reco_stage1_reco_stage2_20240926T141547_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
