Jobsub ID 78462.124@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78462.124@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3454 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:58:37 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 12:16:24 |
| From worker node | Hostname | wn098.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:15:44 |
| Input files | hd-protodune:np04hd_raw_run029108_0266_dataflow5_datawriter_0_20240909T143441.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 12:16:24 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
.
434679 tests to find 227066 neighbours
Iterating with no regularization...
Begin: 5.58344e+06
0 5.07579e+06
1 5.05036e+06
2 5.04547e+06
Now with regularization...
Begin: 4.78096e+06
0 4.77868e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 64085 TrkackID 7 startx, y, z -72.409 596.726 239.309 wireno, peakt tpcno 442 3386.4 6 dist 0min T, max_T2710.33 3633.61
no of hits closeby 0 event 64085 TrkackID 7 startx, y, z -72.409 596.726 239.309 wireno, peakt tpcno 442 3386.4 6 dist 0min T, max_T2710.33 3633.61
Begin processing the 29th record. run: 29108 subRun: 1 event: 64093 at 26-Sep-2024 12:41:30 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 59333 traces, 59333 summary
Retagger: tagging trace set: gauss with 44379 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 44379 traces tagged "gauss"
FrameSaver: q=832826 n=793754 tag=gauss
wclsFrameSaver: saving 59333 traces tagged "wiener"
FrameSaver: q=900112 n=709356 tag=wiener
3218 hits in TPC 0
3737 hits in TPC 1
4343 hits in TPC 2
2054 hits in TPC 3
3699 hits in TPC 4
5279 hits in TPC 5
5861 hits in TPC 6
3262 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 502 XUs and 300 XVs -> 16 XUVs
C:0 T:2 69680 XUs and 91527 XVs -> 2112 XUVs
C:0 T:3 63 XUs and 54 XVs -> 4 XUVs
C:0 T:4 83 XUs and 132 XVs -> 5 XUVs
C:0 T:5 17722 XUs and 10358 XVs -> 188 XUVs
C:0 T:6 75510 XUs and 121780 XVs -> 1901 XUVs
C:0 T:7 157 XUs and 57 XVs -> 1 XUVs
4227 XUVs total
1317 collection wire objects
4227 potential space points
Neighbour search...
307231 tests to find 164392 neighbours
Iterating with no regularization...
Begin: 4.50164e+06
0 4.00905e+06
1 3.97019e+06
2 3.9632e+06
3 3.96035e+06
Now with regularization...
Begin: 3.82877e+06
0 3.82711e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29108 subRun: 1 event: 64101 at 26-Sep-2024 12:43:27 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 56461 traces, 56461 summary
Retagger: tagging trace set: gauss with 41874 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41874 traces tagged "gauss"
FrameSaver: q=891505 n=780481 tag=gauss
wclsFrameSaver: saving 56461 traces tagged "wiener"
FrameSaver: q=955688 n=708219 tag=wiener
2124 hits in TPC 0
1889 hits in TPC 1
4874 hits in TPC 2
2972 hits in TPC 3
2291 hits in TPC 4
3420 hits in TPC 5
6973 hits in TPC 6
4998 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 524 XUs and 334 XVs -> 59 XUVs
C:0 T:2 8713 XUs and 18442 XVs -> 860 XUVs
C:0 T:3 178 XUs and 151 XVs -> 4 XUVs
C:0 T:4 19 XUs and 19 XVs -> 1 XUVs
C:0 T:5 6145 XUs and 10639 XVs -> 255 XUVs
C:0 T:6 34616 XUs and 39394 XVs -> 1133 XUVs
C:0 T:7 663 XUs and 332 XVs -> 17 XUVs
2329 XUVs total
1407 collection wire objects
2329 potential space points
Neighbour search...
61177 tests to find 38930 neighbours
Iterating with no regularization...
Begin: 4.82876e+06
0 4.47511e+06
1 4.46282e+06
2 4.4613e+06
Now with regularization...
Begin: 4.16637e+06
0 4.16574e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 64101 TrkackID 9 startx, y, z 35.179 597.552 40.2906 wireno, peakt tpcno 215 1443.71 2 dist 263.167min T, max_T430.043 1447.5
no of hits closeby 0 event 64101 TrkackID 9 startx, y, z 35.179 597.552 40.2906 wireno, peakt tpcno 215 1443.71 2 dist 263.167min T, max_T430.043 1447.5
26-Sep-2024 12:45:19 BST Closed output file "np04hd_raw_run029108_0266_dataflow5_datawriter_0_20240909T143441_reco_stage1_reco_stage2_20240926T114519_keepup.root"
26-Sep-2024 12:45:19 BST Closed input file "np04hd_raw_run029108_0266_dataflow5_datawriter_0_20240909T143441_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 83.8347 115.299 218.548 113.238 25.9122 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000478007 0.00261719 0.00467151 0.00262261 0.00126816 30
produce:fembfilter:PDHDFEMBFilter 0.718178 0.743437 0.944727 0.731246 0.0408477 30
produce:wclsdatahd:WireCellToolkit 53.053 57.5097 60.6021 58.0103 1.96146 30
produce:gaushit:GausHitFinder 0.424418 0.709536 1.23215 0.686601 0.187735 30
produce:nhitsfilter:NumberOfHitsFilter 0.000250003 0.00042223 0.000754679 0.000407984 0.000115802 30
produce:reco3d:SpacePointSolver 0.154516 1.3085 5.50135 0.849861 1.2128 30
produce:hitpdune:DisambigFromSpacePoints 0.203223 0.569336 2.06967 0.429129 0.418127 30
produce:pandora:StandardPandora 4.3907 13.5505 58.6659 10.9928 10.1014 30
produce:pandoraWriter:StandardPandora 0.0932121 0.15876 0.322662 0.160967 0.0463602 30
produce:pandoraTrack:LArPandoraTrackCreation 0.605827 1.76928 5.26204 1.50525 0.956826 30
produce:pandoraShower:LArPandoraShowerCreation 0.71618 2.09046 9.65278 1.79775 1.65796 30
produce:pandoracalo:Calorimetry 0.330564 0.836195 2.44538 0.761964 0.412709 30
produce:pandoracalonosce:Calorimetry 0.301048 0.775637 2.35988 0.697905 0.403381 30
produce:pandorapid:Chi2ParticleID 0.00130647 0.00195178 0.00337788 0.00186339 0.000410838 30
produce:pandoraShowercalo:ShowerCalorimetry 0.551577 1.81204 7.88608 1.662 1.31362 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.529475 1.77341 7.84971 1.71342 1.31235 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0874153 0.134278 0.222992 0.130266 0.0323196 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0330303 0.0495383 0.0812725 0.0495577 0.0112769 30
produce:emtrkmichelid:EmTrackMichelId 17.4564 29.8394 55.3067 28.994 8.19343 30
[art]:TriggerResults:TriggerResultInserter 2.3793e-05 5.26202e-05 9.6857e-05 4.9696e-05 1.92787e-05 30
end_path:out1:RootOutput 7.473e-06 1.20324e-05 3.3268e-05 1.04355e-05 5.17086e-06 30
end_path:michelremoving:michelremoving 0.0241017 0.0784913 0.222222 0.0502474 0.0567239 30
end_path:michelremovinggnocchi:michelremoving 0.0233848 0.0740087 0.22143 0.0448834 0.0541555 30
end_path:calibanagnocchi:CalibAnaTree 0.182674 0.632601 1.49535 0.631394 0.29547 30
end_path:out1:RootOutput(write) 0.468004 0.745061 1.29168 0.743333 0.185525 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4931.31 MB
Peak resident set size usage (VmHWM): 2344.42 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029108_0266_dataflow5_datawriter_0_20240909T143441_reco_stage1_reco_stage2_20240926T114519_keepup.root
\tHists: np04hd_raw_run029108_0266_dataflow5_datawriter_0_20240909T143441_reco_stage1_reco_stage2_20240926T114519_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
