Jobsub ID 263290.24@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263290.24@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3461 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:57:08 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 11:59:46 |
| From worker node | Hostname | wn086.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2650 v2 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:59:21 |
| Input files | hd-protodune:np04hd_raw_run027298_0028_dataflow1_datawriter_0_20240619T141934.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:43:43 |
| Saved logs | justin-logs:263290.24-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
1 0.000583231 0.000583231 0.000583291 0.000583231 0.000583231 0.000583291 0.000583231 0.000583231 0.000583231 0.000583291
----view 1: 0.0279551 0.0633714 0.0283339 0.0375348 0.0289621 0.03076 0.0319498 0.033489 0.032244 0.0264131 0.0336007
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 34th record. run: 27298 subRun: 1 event: 4053 at 26-Sep-2024 12:39:25 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 49550 traces, 49550 summary
Retagger: tagging trace set: gauss with 39000 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 39000 traces tagged "gauss"
FrameSaver: q=1.44319e+06 n=735070 tag=gauss
wclsFrameSaver: saving 49550 traces tagged "wiener"
FrameSaver: q=1.5268e+06 n=672453 tag=wiener
3021 hits in TPC 0
4107 hits in TPC 1
5033 hits in TPC 2
4267 hits in TPC 3
2319 hits in TPC 4
4347 hits in TPC 5
2265 hits in TPC 6
1559 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 47 XUs and 28 XVs -> 0 XUVs
C:0 T:1 3854 XUs and 2980 XVs -> 92 XUVs
C:0 T:2 20375 XUs and 20699 XVs -> 499 XUVs
C:0 T:3 16 XUs and 16 XVs -> 2 XUVs
C:0 T:4 63 XUs and 63 XVs -> 1 XUVs
C:0 T:5 8572 XUs and 7537 XVs -> 181 XUVs
C:0 T:6 12340 XUs and 5178 XVs -> 165 XUVs
C:0 T:7 30 XUs and 69 XVs -> 1 XUVs
941 XUVs total
747 collection wire objects
941 potential space points
Neighbour search...
7423 tests to find 4370 neighbours
Iterating with no regularization...
Begin: 1.13181e+07
0 1.1086e+07
1 1.10805e+07
Now with regularization...
Begin: 1.03519e+07
0 1.03512e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 4053 TrkackID 3 startx, y, z 315.796 347.491 183.145 wireno, peakt tpcno 311 4021 2 dist 101.772min T, max_T9.31097 4044.75
no of hits closeby 0 event 4053 TrkackID 3 startx, y, z 315.796 347.491 183.145 wireno, peakt tpcno 311 4021 2 dist 101.772min T, max_T9.31097 4044.75
Begin processing the 35th record. run: 27298 subRun: 1 event: 4057 at 26-Sep-2024 12:41:05 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 73954 traces, 73954 summary
Retagger: tagging trace set: gauss with 55227 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 55227 traces tagged "gauss"
FrameSaver: q=2.08294e+06 n=1064287 tag=gauss
wclsFrameSaver: saving 73954 traces tagged "wiener"
FrameSaver: q=2.23211e+06 n=966818 tag=wiener
3648 hits in TPC 0
3892 hits in TPC 1
5914 hits in TPC 2
2977 hits in TPC 3
6652 hits in TPC 4
9746 hits in TPC 5
7041 hits in TPC 6
3579 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2476 XUs and 2350 XVs -> 59 XUVs
C:0 T:2 14769 XUs and 12488 XVs -> 343 XUVs
C:0 T:3 2342 XUs and 1878 XVs -> 66 XUVs
C:0 T:4 310 XUs and 436 XVs -> 6 XUVs
C:0 T:5 43687 XUs and 82648 XVs -> 3214 XUVs
C:0 T:6 12882 XUs and 14442 XVs -> 833 XUVs
C:0 T:7 552 XUs and 795 XVs -> 93 XUVs
4614 XUVs total
1788 collection wire objects
4614 potential space points
Neighbour search...
250834 tests to find 127046 neighbours
Iterating with no regularization...
Begin: 3.73005e+07
0 3.22945e+07
1 3.17578e+07
2 3.17149e+07
3 3.17032e+07
Now with regularization...
Begin: 3.01652e+07
0 3.01588e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 4057 TrkackID 0 startx, y, z 222.992 550.686 0.421891 wireno, peakt tpcno 477 896.103 5 dist 0min T, max_T896.103 4405.2
no of hits closeby 1 event 4057 TrkackID 0 startx, y, z 222.992 550.686 0.421891 wireno, peakt tpcno 477 896.103 5 dist 0min T, max_T896.103 4405.2
26-Sep-2024 12:43:41 BST Closed output file "np04hd_raw_run027298_0028_dataflow1_datawriter_0_20240619T141934_reco_stage1_reco_stage2_20240926T114341_keepup.root"
26-Sep-2024 12:43:41 BST Closed input file "np04hd_raw_run027298_0028_dataflow1_datawriter_0_20240619T141934_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 100.253 134.985 205.429 128.157 23.8438 35
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00045873 0.00256388 0.00476506 0.00269466 0.00124519 35
produce:fembfilter:PDHDFEMBFilter 0.716693 0.811308 1.01226 0.829167 0.0675312 35
produce:wclsdatahd:WireCellToolkit 42.7804 45.5589 60.71 44.4287 3.87674 35
produce:gaushit:GausHitFinder 0.644404 0.94725 1.62555 0.948416 0.232102 35
produce:nhitsfilter:NumberOfHitsFilter 0.000326739 0.000452702 0.000602464 0.000464797 7.20693e-05 35
produce:reco3d:SpacePointSolver 0.280426 0.933466 2.59562 0.92048 0.516886 35
produce:hitpdune:DisambigFromSpacePoints 0.298379 0.837585 1.90013 0.785094 0.354155 35
produce:pandora:StandardPandora 10.0293 19.991 40.7805 18.2464 7.39002 35
produce:pandoraWriter:StandardPandora 0.123917 0.176197 0.267273 0.172341 0.037056 35
produce:pandoraTrack:LArPandoraTrackCreation 0.87434 2.07859 4.9105 1.94354 0.839353 35
produce:pandoraShower:LArPandoraShowerCreation 1.10459 2.37996 5.97355 2.22669 1.24032 35
produce:pandoracalo:Calorimetry 0.520531 1.02671 2.51525 0.969594 0.386301 35
produce:pandoracalonosce:Calorimetry 0.463422 0.942989 2.24075 0.9001 0.348284 35
produce:pandorapid:Chi2ParticleID 0.00141669 0.00235397 0.00461059 0.00219026 0.000691326 35
produce:pandoraShowercalo:ShowerCalorimetry 0.924626 2.30119 5.60608 2.16079 1.12391 35
produce:pandoraShowercalonosce:ShowerCalorimetry 0.909032 2.18671 5.17677 2.11628 1.0369 35
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0913805 0.14629 0.222418 0.14843 0.0315433 35
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0301773 0.0470002 0.0700358 0.0458417 0.00953665 35
produce:emtrkmichelid:EmTrackMichelId 37.4201 52.7029 81.035 49.8086 9.98367 35
[art]:TriggerResults:TriggerResultInserter 2.5638e-05 4.64344e-05 8.1234e-05 4.5188e-05 1.32078e-05 35
end_path:out1:RootOutput 7.198e-06 8.8058e-06 2.495e-05 7.888e-06 3.18015e-06 35
end_path:michelremoving:michelremoving 0.025007 0.0982643 0.420768 0.0418937 0.103045 35
end_path:michelremovinggnocchi:michelremoving 0.0233461 0.0946951 0.415626 0.0401484 0.102308 35
end_path:calibanagnocchi:CalibAnaTree 0.314929 0.759763 1.68564 0.724618 0.257187 35
end_path:out1:RootOutput(write) 0.594886 0.84373 1.26512 0.830654 0.15486 35
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4998.18 MB
Peak resident set size usage (VmHWM): 2411.03 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027298_0028_dataflow1_datawriter_0_20240619T141934_reco_stage1_reco_stage2_20240926T114341_keepup.root
\tHists: np04hd_raw_run027298_0028_dataflow1_datawriter_0_20240619T141934_reco_stage1_reco_stage2_20240926T114341_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
