Jobsub ID 263681.5@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263681.5@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:07:21 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:40:53 |
| From worker node | Hostname | n244.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:08:36 |
| Input files | hd-protodune:np04hd_raw_run029107_2166_dataflow0_datawriter_0_20240908T024602.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:40:53 |
| Saved logs | justin-logs:263681.5-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
l::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 520096 at 26-Sep-2024 15:33:57 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 48709 traces, 48709 summary
Retagger: tagging trace set: gauss with 36334 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 36334 traces tagged "gauss"
FrameSaver: q=793803 n=691263 tag=gauss
wclsFrameSaver: saving 48709 traces tagged "wiener"
FrameSaver: q=847598 n=622983 tag=wiener
2283 hits in TPC 0
2438 hits in TPC 1
3558 hits in TPC 2
2074 hits in TPC 3
3327 hits in TPC 4
6277 hits in TPC 5
4551 hits in TPC 6
2456 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 172 XUs and 49 XVs -> 0 XUVs
C:0 T:2 11767 XUs and 15412 XVs -> 386 XUVs
C:0 T:3 742 XUs and 1055 XVs -> 82 XUVs
C:0 T:4 70 XUs and 125 XVs -> 3 XUVs
C:0 T:5 24358 XUs and 67650 XVs -> 1377 XUVs
C:0 T:6 11860 XUs and 21170 XVs -> 821 XUVs
C:0 T:7 71 XUs and 90 XVs -> 2 XUVs
2671 XUVs total
1041 collection wire objects
2671 potential space points
Neighbour search...
112263 tests to find 61824 neighbours
Iterating with no regularization...
Begin: 5.50149e+06
0 5.02992e+06
1 5.00706e+06
2 5.00493e+06
Now with regularization...
Begin: 4.74661e+06
0 4.74581e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 520096 TrkackID 15 startx, y, z 38.128 561.713 315.86 wireno, peakt tpcno 258 1.0775 5 dist 0min T, max_T1.0775 465.012
no of hits closeby 0 event 520096 TrkackID 15 startx, y, z 38.128 561.713 315.86 wireno, peakt tpcno 258 1.0775 5 dist 0min T, max_T1.0775 465.012
Begin processing the 30th record. run: 29107 subRun: 1 event: 520104 at 26-Sep-2024 15:35:18 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 63470 traces, 63470 summary
Retagger: tagging trace set: gauss with 46638 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 46638 traces tagged "gauss"
FrameSaver: q=976840 n=836972 tag=gauss
wclsFrameSaver: saving 63470 traces tagged "wiener"
FrameSaver: q=1.05139e+06 n=747126 tag=wiener
2622 hits in TPC 0
3582 hits in TPC 1
7145 hits in TPC 2
2655 hits in TPC 3
2689 hits in TPC 4
5515 hits in TPC 5
5019 hits in TPC 6
2748 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 682 XUs and 931 XVs -> 56 XUVs
C:0 T:2 34629 XUs and 39506 XVs -> 1182 XUVs
C:0 T:3 209 XUs and 121 XVs -> 5 XUVs
C:0 T:4 16 XUs and 17 XVs -> 1 XUVs
C:0 T:5 38694 XUs and 27130 XVs -> 1019 XUVs
C:0 T:6 27822 XUs and 32427 XVs -> 813 XUVs
C:0 T:7 113 XUs and 135 XVs -> 2 XUVs
3078 XUVs total
1303 collection wire objects
3078 potential space points
Neighbour search...
137622 tests to find 72974 neighbours
Iterating with no regularization...
Begin: 9.73812e+06
0 9.10174e+06
1 9.0787e+06
2 9.07648e+06
Now with regularization...
Begin: 8.50958e+06
0 8.50855e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 520104 TrkackID 5 startx, y, z 44.0338 594.632 155.79 wireno, peakt tpcno 317 3990.86 2 dist 476.313min T, max_T295 4026.06
no of hits closeby 0 event 520104 TrkackID 6 startx, y, z -49.6474 506.463 170.738 wireno, peakt tpcno 28 1563.74 2 dist 0min T, max_T919.546 1919.58
no of hits closeby 0 event 520104 TrkackID 5 startx, y, z 44.0338 594.632 155.79 wireno, peakt tpcno 317 3990.86 2 dist 476.313min T, max_T295 4026.06
no of hits closeby 0 event 520104 TrkackID 6 startx, y, z -49.6474 506.463 170.738 wireno, peakt tpcno 28 1563.74 2 dist 0min T, max_T919.546 1919.58
26-Sep-2024 15:36:30 BST Closed output file "np04hd_raw_run029107_2166_dataflow0_datawriter_0_20240908T024602_reco_stage1_reco_stage2_20240926T143630_keepup.root"
26-Sep-2024 15:36:30 BST Closed input file "np04hd_raw_run029107_2166_dataflow0_datawriter_0_20240908T024602_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 47.4521 76.1791 107.266 72.6534 14.1034 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00042328 0.00228865 0.00419102 0.0021442 0.00105118 30
produce:fembfilter:PDHDFEMBFilter 0.443935 0.622385 0.682545 0.659367 0.0845989 30
produce:wclsdatahd:WireCellToolkit 26.8172 33.7997 36.6194 34.974 2.96844 30
produce:gaushit:GausHitFinder 0.310275 0.625071 0.979897 0.589289 0.155807 30
produce:nhitsfilter:NumberOfHitsFilter 0.000206861 0.000318158 0.000501307 0.000311334 6.64044e-05 30
produce:reco3d:SpacePointSolver 0.0658977 0.749798 2.27741 0.60722 0.598329 30
produce:hitpdune:DisambigFromSpacePoints 0.101713 0.437496 1.33954 0.379325 0.254082 30
produce:pandora:StandardPandora 2.66043 10.497 24.9242 9.45197 4.66408 30
produce:pandoraWriter:StandardPandora 0.070984 0.141783 0.226767 0.141065 0.0358208 30
produce:pandoraTrack:LArPandoraTrackCreation 0.285386 0.966533 2.08198 0.946505 0.368215 30
produce:pandoraShower:LArPandoraShowerCreation 0.211952 1.02676 2.13066 0.894528 0.445299 30
produce:pandoracalo:Calorimetry 0.165405 0.53825 0.977105 0.522077 0.182905 30
produce:pandoracalonosce:Calorimetry 0.133949 0.479545 0.905423 0.47098 0.176691 30
produce:pandorapid:Chi2ParticleID 0.00101123 0.00184173 0.00442364 0.00174601 0.000643599 30
produce:pandoraShowercalo:ShowerCalorimetry 0.184708 0.996012 2.26205 0.919211 0.490573 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.180698 0.969661 2.07266 0.896902 0.470155 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0612949 0.130652 0.223477 0.129048 0.0413185 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0188049 0.0449807 0.0756645 0.044932 0.0141753 30
produce:emtrkmichelid:EmTrackMichelId 7.7682 22.7238 37.9729 20.7628 6.67745 30
[art]:TriggerResults:TriggerResultInserter 1.7183e-05 3.86542e-05 8.7545e-05 3.3584e-05 1.6329e-05 30
end_path:out1:RootOutput 7.123e-06 1.36426e-05 2.9166e-05 1.3195e-05 4.90999e-06 30
end_path:michelremoving:michelremoving 0.0112709 0.0845247 0.249049 0.0377146 0.0735517 30
end_path:michelremovinggnocchi:michelremoving 0.0101311 0.0823528 0.252685 0.036557 0.0741287 30
end_path:calibanagnocchi:CalibAnaTree 0.139815 0.399261 0.801022 0.393759 0.172596 30
end_path:out1:RootOutput(write) 0.332379 0.722427 1.07508 0.741487 0.20266 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4920.05 MB
Peak resident set size usage (VmHWM): 2596.61 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2166_dataflow0_datawriter_0_20240908T024602_reco_stage1_reco_stage2_20240926T143630_keepup.root
\tHists: np04hd_raw_run029107_2166_dataflow0_datawriter_0_20240908T024602_reco_stage1_reco_stage2_20240926T143630_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
