Jobsub ID 263758.21@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263758.21@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:34 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 15:13:03 |
| From worker node | Hostname | n177.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:39 |
| Input files | hd-protodune:np04hd_raw_run029107_0864_dataflow3_datawriter_0_20240907T044741.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:13:03 |
| Saved logs | justin-logs:263758.21-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
ests to find 66810 neighbours
Iterating with no regularization...
Begin: 1.09251e+07
0 1.01083e+07
1 1.00524e+07
2 1.00486e+07
Now with regularization...
Begin: 9.42786e+06
0 9.42644e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 207627 at 26-Sep-2024 16:09:37 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69623 traces, 69623 summary
Retagger: tagging trace set: gauss with 53001 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 53001 traces tagged "gauss"
FrameSaver: q=1.22673e+06 n=1047964 tag=gauss
wclsFrameSaver: saving 69623 traces tagged "wiener"
FrameSaver: q=1.30556e+06 n=956894 tag=wiener
3598 hits in TPC 0
4574 hits in TPC 1
4483 hits in TPC 2
2580 hits in TPC 3
6353 hits in TPC 4
6325 hits in TPC 5
6736 hits in TPC 6
4062 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 23 XUs and 10 XVs -> 1 XUVs
C:0 T:1 988 XUs and 597 XVs -> 43 XUVs
C:0 T:2 31024 XUs and 34977 XVs -> 1622 XUVs
C:0 T:3 94 XUs and 97 XVs -> 0 XUVs
C:0 T:4 530 XUs and 760 XVs -> 38 XUVs
C:0 T:5 24959 XUs and 29463 XVs -> 770 XUVs
C:0 T:6 35425 XUs and 45398 XVs -> 887 XUVs
C:0 T:7 531 XUs and 964 XVs -> 51 XUVs
3412 XUVs total
1575 collection wire objects
3412 potential space points
Neighbour search...
225088 tests to find 108750 neighbours
Iterating with no regularization...
Begin: 8.94944e+06
0 8.54364e+06
1 8.52105e+06
2 8.5184e+06
Now with regularization...
Begin: 8.06832e+06
0 8.06738e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 207635 at 26-Sep-2024 16:11:14 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58186 traces, 58186 summary
Retagger: tagging trace set: gauss with 43550 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43550 traces tagged "gauss"
FrameSaver: q=955110 n=806357 tag=gauss
wclsFrameSaver: saving 58186 traces tagged "wiener"
FrameSaver: q=1.02578e+06 n=733984 tag=wiener
2344 hits in TPC 0
3210 hits in TPC 1
3436 hits in TPC 2
2407 hits in TPC 3
4143 hits in TPC 4
6099 hits in TPC 5
5235 hits in TPC 6
3284 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 487 XUs and 353 XVs -> 3 XUVs
C:0 T:2 8303 XUs and 7922 XVs -> 340 XUVs
C:0 T:3 227 XUs and 401 XVs -> 21 XUVs
C:0 T:4 94 XUs and 93 XVs -> 3 XUVs
C:0 T:5 35936 XUs and 47428 XVs -> 3170 XUVs
C:0 T:6 12540 XUs and 14437 XVs -> 1100 XUVs
C:0 T:7 5 XUs and 13 XVs -> 0 XUVs
4637 XUVs total
1249 collection wire objects
4637 potential space points
Neighbour search...
524395 tests to find 243590 neighbours
Iterating with no regularization...
Begin: 6.02783e+06
0 5.59089e+06
1 5.56043e+06
2 5.55391e+06
3 5.55181e+06
Now with regularization...
Begin: 5.23825e+06
0 5.2369e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 207635 TrkackID 8 startx, y, z 73.7592 592.927 372.939 wireno, peakt tpcno 322 3046.28 5 dist 0min T, max_T3043.3 4453.5
no of hits closeby 4 event 207635 TrkackID 9 startx, y, z 48.7211 589.725 342.66 wireno, peakt tpcno 441 3531.82 5 dist 0min T, max_T3528.9 4529.43
no of hits closeby 0 event 207635 TrkackID 8 startx, y, z 73.7592 592.927 372.939 wireno, peakt tpcno 322 3046.28 5 dist 0min T, max_T3043.3 4453.5
no of hits closeby 4 event 207635 TrkackID 9 startx, y, z 48.7211 589.725 342.66 wireno, peakt tpcno 441 3531.82 5 dist 0min T, max_T3528.9 4529.43
26-Sep-2024 16:12:35 BST Closed output file "np04hd_raw_run029107_0864_dataflow3_datawriter_0_20240907T044741_reco_stage1_reco_stage2_20240926T151235_keepup.root"
26-Sep-2024 16:12:35 BST Closed input file "np04hd_raw_run029107_0864_dataflow3_datawriter_0_20240907T044741_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 64.1645 109.146 160.343 107.501 22.1819 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00038822 0.00161845 0.00485381 0.00133124 0.00111755 30
produce:fembfilter:PDHDFEMBFilter 0.691199 0.878079 1.17087 0.757505 0.187943 30
produce:wclsdatahd:WireCellToolkit 44.0661 52.5086 65.9925 48.8092 8.01298 30
produce:gaushit:GausHitFinder 0.36241 1.01877 2.1677 1.01227 0.362259 30
produce:nhitsfilter:NumberOfHitsFilter 0.000237122 0.000436062 0.00068216 0.000432061 0.000108086 30
produce:reco3d:SpacePointSolver 0.304385 1.5367 3.97173 1.43525 1.01356 30
produce:hitpdune:DisambigFromSpacePoints 0.142357 0.771856 1.74791 0.791539 0.345303 30
produce:pandora:StandardPandora 5.1452 16.8415 34.0513 16.3388 6.48889 30
produce:pandoraWriter:StandardPandora 0.0902025 0.184983 0.450397 0.173241 0.0659772 30
produce:pandoraTrack:LArPandoraTrackCreation 0.469436 2.25967 7.58387 1.88093 1.34121 30
produce:pandoraShower:LArPandoraShowerCreation 0.420251 2.5289 8.18027 2.04053 1.63291 30
produce:pandoracalo:Calorimetry 0.214985 0.960949 2.66031 0.811535 0.534918 30
produce:pandoracalonosce:Calorimetry 0.192529 0.903598 2.70242 0.743015 0.537654 30
produce:pandorapid:Chi2ParticleID 0.000986849 0.00223794 0.00390635 0.00205151 0.000738865 30
produce:pandoraShowercalo:ShowerCalorimetry 0.322145 2.16961 7.21181 1.75849 1.3828 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.316625 2.10634 5.55285 1.82155 1.21943 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0584153 0.157475 0.281369 0.145826 0.0518502 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0260788 0.0605961 0.113575 0.0547497 0.0203946 30
produce:emtrkmichelid:EmTrackMichelId 9.97029 22.0323 40.6388 21.5227 6.12975 30
[art]:TriggerResults:TriggerResultInserter 2.636e-05 5.13094e-05 8.2242e-05 5.0287e-05 1.35152e-05 30
end_path:out1:RootOutput 7.469e-06 2.08999e-05 0.0002222 1.35295e-05 3.77353e-05 30
end_path:michelremoving:michelremoving 0.0202648 0.1066 0.688497 0.059748 0.136003 30
end_path:michelremovinggnocchi:michelremoving 0.0176243 0.102542 0.67167 0.0520977 0.132646 30
end_path:calibanagnocchi:CalibAnaTree 0.19955 0.769543 2.49812 0.639639 0.473971 30
end_path:out1:RootOutput(write) 0.607621 1.05248 2.06563 0.960451 0.350574 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5772.09 MB
Peak resident set size usage (VmHWM): 2798.31 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0864_dataflow3_datawriter_0_20240907T044741_reco_stage1_reco_stage2_20240926T151235_keepup.root
\tHists: np04hd_raw_run029107_0864_dataflow3_datawriter_0_20240907T044741_reco_stage1_reco_stage2_20240926T151235_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
