Jobsub ID 263940.103@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263940.103@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:25:09 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 15:37:11 |
| From worker node | Hostname | wn031.hep |
| cpuinfo | Intel(R) Xeon(R) Gold 6226 CPU @ 2.70GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:27:07 |
| Input files | hd-protodune:np04hd_raw_run029097_0265_dataflow2_datawriter_0_20240906T155022.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:37:09 |
| Saved logs | justin-logs:263940.103-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
ces...
C:0 T:1 278 XUs and 119 XVs -> 5 XUVs
C:0 T:2 21682 XUs and 78805 XVs -> 2601 XUVs
C:0 T:3 54 XUs and 91 XVs -> 3 XUVs
C:0 T:4 51 XUs and 85 XVs -> 7 XUVs
C:0 T:5 19952 XUs and 18775 XVs -> 378 XUVs
C:0 T:6 33303 XUs and 56907 XVs -> 3565 XUVs
C:0 T:7 116 XUs and 312 XVs -> 9 XUVs
6568 XUVs total
1545 collection wire objects
6568 potential space points
Neighbour search...
739832 tests to find 347430 neighbours
Iterating with no regularization...
Begin: 9.87604e+06
0 8.61072e+06
1 8.49184e+06
2 8.47755e+06
3 8.47353e+06
Now with regularization...
Begin: 8.18933e+06
0 8.18716e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 63818 TrkackID 1 startx, y, z 17.8158 473.541 9.9385 wireno, peakt tpcno 450 283.465 5 dist 0min T, max_T175.967 418.472
no of hits closeby 1 event 63818 TrkackID 1 startx, y, z 17.8158 473.541 9.9385 wireno, peakt tpcno 450 283.465 5 dist 0min T, max_T175.967 418.472
Begin processing the 29th record. run: 29097 subRun: 1 event: 63826 at 26-Sep-2024 16:33:12 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 56730 traces, 56730 summary
Retagger: tagging trace set: gauss with 41892 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41892 traces tagged "gauss"
FrameSaver: q=900004 n=749448 tag=gauss
wclsFrameSaver: saving 56730 traces tagged "wiener"
FrameSaver: q=966253 n=672827 tag=wiener
2672 hits in TPC 0
2232 hits in TPC 1
4967 hits in TPC 2
3377 hits in TPC 3
2576 hits in TPC 4
5452 hits in TPC 5
5061 hits in TPC 6
1963 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 332 XUs and 203 XVs -> 2 XUVs
C:0 T:2 14656 XUs and 24167 XVs -> 1186 XUVs
C:0 T:3 771 XUs and 1306 XVs -> 36 XUVs
C:0 T:4 274 XUs and 471 XVs -> 31 XUVs
C:0 T:5 14048 XUs and 14380 XVs -> 258 XUVs
C:0 T:6 7158 XUs and 22185 XVs -> 300 XUVs
C:0 T:7 66 XUs and 80 XVs -> 1 XUVs
1814 XUVs total
995 collection wire objects
1814 potential space points
Neighbour search...
36814 tests to find 20352 neighbours
Iterating with no regularization...
Begin: 4.23939e+06
0 3.85486e+06
1 3.82499e+06
2 3.82234e+06
Now with regularization...
Begin: 3.4506e+06
0 3.44995e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 63834 at 26-Sep-2024 16:34:31 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69889 traces, 69889 summary
Retagger: tagging trace set: gauss with 53482 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 53482 traces tagged "gauss"
FrameSaver: q=1.20534e+06 n=1093607 tag=gauss
wclsFrameSaver: saving 69889 traces tagged "wiener"
FrameSaver: q=1.28801e+06 n=1001282 tag=wiener
4144 hits in TPC 0
4071 hits in TPC 1
8552 hits in TPC 2
6536 hits in TPC 3
2246 hits in TPC 4
3600 hits in TPC 5
9747 hits in TPC 6
4034 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 100 XUs and 52 XVs -> 0 XUVs
C:0 T:2 32946 XUs and 72075 XVs -> 2085 XUVs
C:0 T:3 513 XUs and 647 XVs -> 17 XUVs
C:0 T:4 59 XUs and 122 XVs -> 9 XUVs
C:0 T:5 5717 XUs and 6098 XVs -> 181 XUVs
C:0 T:6 37283 XUs and 38391 XVs -> 1035 XUVs
C:0 T:7 538 XUs and 582 XVs -> 26 XUVs
3353 XUVs total
1772 collection wire objects
3353 potential space points
Neighbour search...
143679 tests to find 65176 neighbours
Iterating with no regularization...
Begin: 1.40472e+07
0 1.25542e+07
1 1.24309e+07
2 1.24242e+07
Now with regularization...
Begin: 1.17814e+07
0 1.17783e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:36:48 BST Closed output file "np04hd_raw_run029097_0265_dataflow2_datawriter_0_20240906T155022_reco_stage1_reco_stage2_20240926T153648_keepup.root"
26-Sep-2024 16:36:48 BST Closed input file "np04hd_raw_run029097_0265_dataflow2_datawriter_0_20240906T155022_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 68.9265 89.3754 136.458 83.7663 16.4017 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000337817 0.00127141 0.00376439 0.00103567 0.000874264 30
produce:fembfilter:PDHDFEMBFilter 0.598936 0.621913 0.63966 0.62112 0.0104708 30
produce:wclsdatahd:WireCellToolkit 44.5227 48.0275 50.7037 48.1546 1.60278 30
produce:gaushit:GausHitFinder 0.331849 0.655478 1.02733 0.676366 0.163966 30
produce:nhitsfilter:NumberOfHitsFilter 0.000245945 0.000399005 0.000583129 0.00038083 7.83294e-05 30
produce:reco3d:SpacePointSolver 0.174567 0.872878 2.91027 0.611328 0.641165 30
produce:hitpdune:DisambigFromSpacePoints 0.198207 0.500509 1.09574 0.438805 0.237206 30
produce:pandora:StandardPandora 5.19801 11.8985 26.1415 10.6984 5.37801 30
produce:pandoraWriter:StandardPandora 0.101209 0.149723 0.258215 0.141222 0.0362613 30
produce:pandoraTrack:LArPandoraTrackCreation 0.853411 1.97963 5.83728 1.7127 1.04304 30
produce:pandoraShower:LArPandoraShowerCreation 0.852686 2.27953 7.31308 1.73896 1.50364 30
produce:pandoracalo:Calorimetry 0.361912 0.828445 2.30399 0.75919 0.389778 30
produce:pandoracalonosce:Calorimetry 0.337476 0.776504 2.24199 0.702396 0.380855 30
produce:pandorapid:Chi2ParticleID 0.00116666 0.00164233 0.00292662 0.00158746 0.000372161 30
produce:pandoraShowercalo:ShowerCalorimetry 0.66646 1.94538 5.27025 1.64884 1.15891 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.668485 1.9189 5.15717 1.56885 1.14033 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.072416 0.110027 0.15647 0.104708 0.0229747 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0294117 0.0447128 0.0655205 0.042903 0.00950753 30
produce:emtrkmichelid:EmTrackMichelId 9.4261 15.1974 26.0816 14.275 4.00519 30
[art]:TriggerResults:TriggerResultInserter 2.6455e-05 4.10613e-05 8.0302e-05 3.9909e-05 1.06409e-05 30
end_path:out1:RootOutput 6.586e-06 9.79877e-06 2.3732e-05 9.3615e-06 3.17052e-06 30
end_path:michelremoving:michelremoving 0.0226393 0.0546892 0.165964 0.0382329 0.036335 30
end_path:michelremovinggnocchi:michelremoving 0.0209773 0.0517992 0.164954 0.0355805 0.0368518 30
end_path:calibanagnocchi:CalibAnaTree 0.273961 0.594517 1.62973 0.495389 0.289685 30
end_path:out1:RootOutput(write) 0.430647 0.667568 1.01941 0.652819 0.15275 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 6070.98 MB
Peak resident set size usage (VmHWM): 2777.84 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0265_dataflow2_datawriter_0_20240906T155022_reco_stage1_reco_stage2_20240926T153648_keepup.root
\tHists: np04hd_raw_run029097_0265_dataflow2_datawriter_0_20240906T155022_reco_stage1_reco_stage2_20240926T153648_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
