Jobsub ID 78830.16@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78830.16@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:07:16 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:36:54 |
| From worker node | Hostname | n258.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:08:33 |
| Input files | hd-protodune:np04hd_raw_run029107_0754_dataflow4_datawriter_0_20240907T032011.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:36:54 |
| Saved logs | justin-logs:78830.16-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
th regularization...
Begin: 4.66058e+06
0 4.65978e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 181212 at 26-Sep-2024 15:31:27 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 44554 traces, 44554 summary
Retagger: tagging trace set: gauss with 34143 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 34143 traces tagged "gauss"
FrameSaver: q=720347 n=665628 tag=gauss
wclsFrameSaver: saving 44554 traces tagged "wiener"
FrameSaver: q=769206 n=610569 tag=wiener
3504 hits in TPC 0
3434 hits in TPC 1
3242 hits in TPC 2
2912 hits in TPC 3
1935 hits in TPC 4
5233 hits in TPC 5
3888 hits in TPC 6
1569 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2026 XUs and 767 XVs -> 36 XUVs
C:0 T:2 5536 XUs and 6680 XVs -> 210 XUVs
C:0 T:3 388 XUs and 296 XVs -> 11 XUVs
C:0 T:4 15 XUs and 18 XVs -> 0 XUVs
C:0 T:5 15333 XUs and 6973 XVs -> 414 XUVs
C:0 T:6 9277 XUs and 13439 XVs -> 320 XUVs
C:0 T:7 136 XUs and 128 XVs -> 4 XUVs
995 XUVs total
697 collection wire objects
995 potential space points
Neighbour search...
12629 tests to find 7982 neighbours
Iterating with no regularization...
Begin: 3.23604e+06
0 3.14883e+06
1 3.14436e+06
2 3.14397e+06
Now with regularization...
Begin: 2.98953e+06
0 2.98937e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 5 event 181212 TrkackID 9 startx, y, z 32.0684 181.773 321.886 wireno, peakt tpcno 167 11.5 5 dist 0min T, max_T4.17579 416.5
no of hits closeby 5 event 181212 TrkackID 9 startx, y, z 32.0684 181.773 321.886 wireno, peakt tpcno 167 11.5 5 dist 0min T, max_T4.17579 416.5
Begin processing the 30th record. run: 29107 subRun: 1 event: 181220 at 26-Sep-2024 15:32:39 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 97912 traces, 97912 summary
Retagger: tagging trace set: gauss with 71183 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 71183 traces tagged "gauss"
FrameSaver: q=1.50437e+06 n=1286503 tag=gauss
wclsFrameSaver: saving 97912 traces tagged "wiener"
FrameSaver: q=1.62606e+06 n=1149620 tag=wiener
5159 hits in TPC 0
3934 hits in TPC 1
12627 hits in TPC 2
4474 hits in TPC 3
2976 hits in TPC 4
5436 hits in TPC 5
10308 hits in TPC 6
5432 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2270 XUs and 1852 XVs -> 47 XUVs
C:0 T:2 72527 XUs and 102395 XVs -> 4828 XUVs
C:0 T:3 199 XUs and 359 XVs -> 5 XUVs
C:0 T:4 24 XUs and 52 XVs -> 1 XUVs
C:0 T:5 14056 XUs and 14691 XVs -> 370 XUVs
C:0 T:6 62989 XUs and 85824 XVs -> 3254 XUVs
C:0 T:7 429 XUs and 666 XVs -> 8 XUVs
8513 XUVs total
2492 collection wire objects
8513 potential space points
Neighbour search...
722723 tests to find 337590 neighbours
Iterating with no regularization...
Begin: 1.38886e+07
0 1.24636e+07
1 1.23002e+07
2 1.22696e+07
3 1.22599e+07
Now with regularization...
Begin: 1.15632e+07
0 1.15575e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 181220 TrkackID 21 startx, y, z 24.2301 180.626 453.951 wireno, peakt tpcno 177 9.0721 5 dist 0min T, max_T8.16518 742
no of hits closeby 3 event 181220 TrkackID 33 startx, y, z 21.8429 374.025 11.9954 wireno, peakt tpcno 42 998.929 2 dist 81.0815min T, max_T388 1060
no of hits closeby 0 event 181220 TrkackID 21 startx, y, z 24.2301 180.626 453.951 wireno, peakt tpcno 177 9.0721 5 dist 0min T, max_T8.16518 742
no of hits closeby 3 event 181220 TrkackID 33 startx, y, z 21.8429 374.025 11.9954 wireno, peakt tpcno 42 998.929 2 dist 81.0815min T, max_T388 1060
26-Sep-2024 15:34:40 BST Closed output file "np04hd_raw_run029107_0754_dataflow4_datawriter_0_20240907T032011_reco_stage1_reco_stage2_20240926T143440_keepup.root"
26-Sep-2024 15:34:40 BST Closed input file "np04hd_raw_run029107_0754_dataflow4_datawriter_0_20240907T032011_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 46.1658 77.6538 115.936 75.7385 16.0943 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000417727 0.00195657 0.00370364 0.00197973 0.000838588 30
produce:fembfilter:PDHDFEMBFilter 0.445462 0.642889 0.883968 0.660813 0.0861018 30
produce:wclsdatahd:WireCellToolkit 25.1226 34.3908 37.2598 35.7654 3.4903 30
produce:gaushit:GausHitFinder 0.301941 0.709534 1.08296 0.70766 0.182646 30
produce:nhitsfilter:NumberOfHitsFilter 0.000194527 0.000320201 0.000480546 0.000313672 6.70451e-05 30
produce:reco3d:SpacePointSolver 0.128635 0.855148 2.18128 0.573516 0.626748 30
produce:hitpdune:DisambigFromSpacePoints 0.123834 0.536459 1.38786 0.516068 0.286711 30
produce:pandora:StandardPandora 3.71131 11.7926 21.5252 10.5728 4.92162 30
produce:pandoraWriter:StandardPandora 0.062422 0.1438 0.242982 0.155949 0.0436373 30
produce:pandoraTrack:LArPandoraTrackCreation 0.323356 0.924013 1.89352 0.841724 0.40884 30
produce:pandoraShower:LArPandoraShowerCreation 0.259396 1.09382 2.56921 0.976851 0.555858 30
produce:pandoracalo:Calorimetry 0.190898 0.529736 0.971564 0.524522 0.194964 30
produce:pandoracalonosce:Calorimetry 0.168218 0.470474 0.908596 0.46462 0.181605 30
produce:pandorapid:Chi2ParticleID 0.000929653 0.00180553 0.00362319 0.00177137 0.000492343 30
produce:pandoraShowercalo:ShowerCalorimetry 0.246863 1.04267 2.44697 1.02849 0.533481 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.242877 1.01759 2.42548 1.00189 0.523405 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0549706 0.147047 0.242906 0.154707 0.0405929 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0191166 0.0482064 0.0806848 0.0477555 0.0137674 30
produce:emtrkmichelid:EmTrackMichelId 10.8217 21.9381 36.8763 21.1111 6.34552 30
[art]:TriggerResults:TriggerResultInserter 2.119e-05 3.076e-05 5.7008e-05 2.9175e-05 7.5243e-06 30
end_path:out1:RootOutput 7.774e-06 1.72302e-05 0.000211299 1.0039e-05 3.61195e-05 30
end_path:michelremoving:michelremoving 0.0155732 0.0732066 0.432887 0.0281622 0.0910294 30
end_path:michelremovinggnocchi:michelremoving 0.0126687 0.066906 0.423565 0.0257722 0.0865965 30
end_path:calibanagnocchi:CalibAnaTree 0.139536 0.370116 0.793346 0.37053 0.141698 30
end_path:out1:RootOutput(write) 0.38512 0.754254 1.19474 0.73945 0.185137 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4933.57 MB
Peak resident set size usage (VmHWM): 2606.19 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0754_dataflow4_datawriter_0_20240907T032011_reco_stage1_reco_stage2_20240926T143440_keepup.root
\tHists: np04hd_raw_run029107_0754_dataflow4_datawriter_0_20240907T032011_reco_stage1_reco_stage2_20240926T143440_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
