Jobsub ID 78692.180@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78692.180@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 11:28:52 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce01 |
| Last heartbeat | 2024-09-26 14:16:41 |
| From worker node | Hostname | wn1907291.tier2.hep.manchester.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:19:15 |
| Input files | hd-protodune:np04hd_raw_run029092_0356_dataflow2_datawriter_0_20240905T225353.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:16:41 |
| Saved logs | justin-logs:78692.180-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
53 0.0177698 0.0183525 0.00970078
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 6.27637e-05 6.27637e-05 6.27637e-05 6.27637e-05 6.27041e-05 6.28233e-05 6.27041e-05 6.28233e-05 6.27041e-05 6.28233e-05 6.27041e-05
----view 1: 0.016798 0.0141373 0.00982666 0.00727654 0.0209942 0.00708199 0.0197153 0.0177698 0.0183525 0.00970078 0.0129242
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 85658 TrkackID 28 startx, y, z 72.4123 601.453 454.76 wireno, peakt tpcno 424 3518.56 5 dist 0min T, max_T2726.96 3620.5
no of hits closeby 2 event 85658 TrkackID 28 startx, y, z 72.4123 601.453 454.76 wireno, peakt tpcno 424 3518.56 5 dist 0min T, max_T2726.96 3620.5
Begin processing the 29th record. run: 29092 subRun: 1 event: 85666 at 26-Sep-2024 15:10:44 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65590 traces, 65590 summary
Retagger: tagging trace set: gauss with 48497 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48497 traces tagged "gauss"
FrameSaver: q=1.03663e+06 n=913566 tag=gauss
wclsFrameSaver: saving 65590 traces tagged "wiener"
FrameSaver: q=1.11643e+06 n=825376 tag=wiener
4439 hits in TPC 0
4301 hits in TPC 1
6267 hits in TPC 2
3133 hits in TPC 3
3725 hits in TPC 4
6233 hits in TPC 5
4873 hits in TPC 6
2005 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 269 XUs and 564 XVs -> 26 XUVs
C:0 T:2 46067 XUs and 39151 XVs -> 1486 XUVs
C:0 T:3 12 XUs and 12 XVs -> 1 XUVs
C:0 T:4 299 XUs and 264 XVs -> 9 XUVs
C:0 T:5 16318 XUs and 33563 XVs -> 1278 XUVs
C:0 T:6 37293 XUs and 49009 XVs -> 2767 XUVs
C:0 T:7 208 XUs and 145 XVs -> 4 XUVs
5571 XUVs total
1564 collection wire objects
5571 potential space points
Neighbour search...
549067 tests to find 258922 neighbours
Iterating with no regularization...
Begin: 7.38232e+06
0 6.63463e+06
1 6.58604e+06
2 6.57894e+06
3 6.57644e+06
Now with regularization...
Begin: 6.1362e+06
0 6.1345e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29092 subRun: 1 event: 85674 at 26-Sep-2024 15:13:30 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65508 traces, 65508 summary
Retagger: tagging trace set: gauss with 46225 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 46225 traces tagged "gauss"
FrameSaver: q=945113 n=823912 tag=gauss
wclsFrameSaver: saving 65508 traces tagged "wiener"
FrameSaver: q=1.04008e+06 n=740675 tag=wiener
3481 hits in TPC 0
3443 hits in TPC 1
5803 hits in TPC 2
2363 hits in TPC 3
3204 hits in TPC 4
4871 hits in TPC 5
6668 hits in TPC 6
2818 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 487 XUs and 608 XVs -> 10 XUVs
C:0 T:2 50663 XUs and 69631 XVs -> 1447 XUVs
C:0 T:3 31 XUs and 75 XVs -> 1 XUVs
C:0 T:4 232 XUs and 213 XVs -> 6 XUVs
C:0 T:5 11470 XUs and 28779 XVs -> 1429 XUVs
C:0 T:6 52979 XUs and 107504 XVs -> 3032 XUVs
C:0 T:7 80 XUs and 81 XVs -> 4 XUVs
5929 XUVs total
1673 collection wire objects
5929 potential space points
Neighbour search...
671079 tests to find 304362 neighbours
Iterating with no regularization...
Begin: 1.02408e+07
0 9.33835e+06
1 9.25646e+06
2 9.24566e+06
3 9.24128e+06
Now with regularization...
Begin: 8.94328e+06
0 8.9407e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 85674 TrkackID 19 startx, y, z 63.4594 591.677 411.959 wireno, peakt tpcno 450 1629.4 5 dist 0min T, max_T1029.71 1827.02
no of hits closeby 0 event 85674 TrkackID 19 startx, y, z 63.4594 591.677 411.959 wireno, peakt tpcno 450 1629.4 5 dist 0min T, max_T1029.71 1827.02
26-Sep-2024 15:16:08 BST Closed output file "np04hd_raw_run029092_0356_dataflow2_datawriter_0_20240905T225353_reco_stage1_reco_stage2_20240926T141608_keepup.root"
26-Sep-2024 15:16:08 BST Closed input file "np04hd_raw_run029092_0356_dataflow2_datawriter_0_20240905T225353_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 101.659 164.338 219.716 165.214 22.6509 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00115332 0.00361673 0.0115636 0.00352368 0.0019576 30
produce:fembfilter:PDHDFEMBFilter 0.729601 1.10395 1.42451 1.10614 0.133994 30
produce:wclsdatahd:WireCellToolkit 64.0479 68.6304 74.8972 68.2068 2.94702 30
produce:gaushit:GausHitFinder 0.681582 1.17438 1.79166 1.16357 0.268309 30
produce:nhitsfilter:NumberOfHitsFilter 0.000359589 0.000653876 0.00307005 0.000554556 0.000481696 30
produce:reco3d:SpacePointSolver 0.173935 1.68758 5.20423 1.29313 1.25656 30
produce:hitpdune:DisambigFromSpacePoints 0.208814 0.886284 1.65716 0.837794 0.344258 30
produce:pandora:StandardPandora 6.43636 19.5238 41.9252 19.6792 6.30253 30
produce:pandoraWriter:StandardPandora 0.0880522 0.250779 0.328251 0.266473 0.0489429 30
produce:pandoraTrack:LArPandoraTrackCreation 0.359279 2.96488 4.95923 2.91932 1.07032 30
produce:pandoraShower:LArPandoraShowerCreation 0.537811 3.34735 7.38699 3.21758 1.44392 30
produce:pandoracalo:Calorimetry 0.346782 1.30245 2.31011 1.25889 0.412946 30
produce:pandoracalonosce:Calorimetry 0.18475 1.17041 2.14981 1.10553 0.409258 30
produce:pandorapid:Chi2ParticleID 0.00103644 0.00302695 0.00482671 0.0030998 0.000759769 30
produce:pandoraShowercalo:ShowerCalorimetry 0.356486 2.83857 5.81726 2.73049 1.15662 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.350895 2.70529 5.40943 2.63046 1.11331 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0838131 0.250277 0.342464 0.254546 0.0516253 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0291277 0.102346 0.220885 0.103892 0.0303494 30
produce:emtrkmichelid:EmTrackMichelId 25.1606 53.5298 66.9966 56.8748 9.42716 30
[art]:TriggerResults:TriggerResultInserter 4.3177e-05 6.41546e-05 0.000105494 6.1849e-05 1.17627e-05 30
end_path:out1:RootOutput 9.377e-06 1.53516e-05 3.273e-05 1.45685e-05 4.02568e-06 30
end_path:michelremoving:michelremoving 0.0390795 0.156928 0.544217 0.0666768 0.136661 30
end_path:michelremovinggnocchi:michelremoving 0.0391782 0.154827 0.538423 0.0633321 0.139022 30
end_path:calibanagnocchi:CalibAnaTree 0.327197 0.982316 1.83348 1.0129 0.304691 30
end_path:out1:RootOutput(write) 0.833081 1.3747 2.04836 1.352 0.263605 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4879.9 MB
Peak resident set size usage (VmHWM): 2554.85 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0356_dataflow2_datawriter_0_20240905T225353_reco_stage1_reco_stage2_20240926T141608_keepup.root
\tHists: np04hd_raw_run029092_0356_dataflow2_datawriter_0_20240905T225353_reco_stage1_reco_stage2_20240926T141608_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
