Jobsub ID 78692.169@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78692.169@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 11:28:52 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce01 |
| Last heartbeat | 2024-09-26 14:14:23 |
| From worker node | Hostname | wn1908240.tier2.hep.manchester.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:19:58 |
| Input files | hd-protodune:np04hd_raw_run029081_0679_dataflow0_datawriter_0_20240905T035041.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:14:23 |
| Saved logs | justin-logs:78692.169-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
8 XUs and 187 XVs -> 9 XUVs
C:0 T:4 367 XUs and 412 XVs -> 23 XUVs
C:0 T:5 29499 XUs and 23370 XVs -> 873 XUVs
C:0 T:6 6488 XUs and 6254 XVs -> 683 XUVs
2568 XUVs total
1340 collection wire objects
2568 potential space points
Neighbour search...
57086 tests to find 32122 neighbours
Iterating with no regularization...
Begin: 1.39848e+07
0 1.36015e+07
1 1.35828e+07
2 1.35808e+07
Now with regularization...
Begin: 1.28534e+07
0 1.28526e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 163184 TrkackID 16 startx, y, z 49.6986 595.691 123.082 wireno, peakt tpcno 99999 -1 -1 dist 99999min T, max_T16 1157.99
no of hits closeby 0 event 163184 TrkackID 16 startx, y, z 49.6986 595.691 123.082 wireno, peakt tpcno 99999 -1 -1 dist 99999min T, max_T16 1157.99
Begin processing the 29th record. run: 29081 subRun: 1 event: 163192 at 26-Sep-2024 15:08:49 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69117 traces, 69117 summary
Retagger: tagging trace set: gauss with 51194 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51194 traces tagged "gauss"
FrameSaver: q=1.09194e+06 n=957697 tag=gauss
wclsFrameSaver: saving 69117 traces tagged "wiener"
FrameSaver: q=1.18445e+06 n=872459 tag=wiener
4477 hits in TPC 0
4401 hits in TPC 1
5898 hits in TPC 2
3505 hits in TPC 3
3483 hits in TPC 4
4990 hits in TPC 5
6059 hits in TPC 6
3550 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1752 XUs and 2386 XVs -> 243 XUVs
C:0 T:2 51723 XUs and 105239 XVs -> 3337 XUVs
C:0 T:3 31 XUs and 50 XVs -> 1 XUVs
C:0 T:4 63 XUs and 78 XVs -> 4 XUVs
C:0 T:5 8903 XUs and 16568 XVs -> 391 XUVs
C:0 T:6 42667 XUs and 57998 XVs -> 2347 XUVs
C:0 T:7 72 XUs and 150 XVs -> 4 XUVs
6327 XUVs total
1738 collection wire objects
6327 potential space points
Neighbour search...
583655 tests to find 271618 neighbours
Iterating with no regularization...
Begin: 1.23851e+07
0 1.09559e+07
1 1.08955e+07
2 1.08847e+07
Now with regularization...
Begin: 1.04603e+07
0 1.04556e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 163200 at 26-Sep-2024 15:11:29 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 55746 traces, 55746 summary
Retagger: tagging trace set: gauss with 40604 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40604 traces tagged "gauss"
FrameSaver: q=844714 n=754112 tag=gauss
wclsFrameSaver: saving 55746 traces tagged "wiener"
FrameSaver: q=903462 n=677211 tag=wiener
2552 hits in TPC 0
2730 hits in TPC 1
5535 hits in TPC 2
2880 hits in TPC 3
1613 hits in TPC 4
3069 hits in TPC 5
5928 hits in TPC 6
3916 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 256 XUs and 413 XVs -> 53 XUVs
C:0 T:2 21729 XUs and 17845 XVs -> 373 XUVs
C:0 T:3 84 XUs and 75 XVs -> 3 XUVs
C:0 T:4 113 XUs and 174 XVs -> 5 XUVs
C:0 T:5 2935 XUs and 3417 XVs -> 230 XUVs
C:0 T:6 20026 XUs and 18533 XVs -> 459 XUVs
C:0 T:7 350 XUs and 313 XVs -> 7 XUVs
1130 XUVs total
759 collection wire objects
1130 potential space points
Neighbour search...
12704 tests to find 7256 neighbours
Iterating with no regularization...
Begin: 3.75729e+06
0 3.636e+06
1 3.632e+06
2 3.63164e+06
Now with regularization...
Begin: 3.40706e+06
0 3.40698e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:13:47 BST Closed output file "np04hd_raw_run029081_0679_dataflow0_datawriter_0_20240905T035041_reco_stage1_reco_stage2_20240926T141347_keepup.root"
26-Sep-2024 15:13:47 BST Closed input file "np04hd_raw_run029081_0679_dataflow0_datawriter_0_20240905T035041_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 105.918 163.707 253.902 159.69 32.9033 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000790303 0.00393283 0.0280418 0.00318965 0.0046151 30
produce:fembfilter:PDHDFEMBFilter 0.735307 1.14705 1.9281 1.1195 0.202518 30
produce:wclsdatahd:WireCellToolkit 62.9248 71.6933 80.859 70.9815 5.39766 30
produce:gaushit:GausHitFinder 0.593628 1.12518 2.21588 1.14042 0.305786 30
produce:nhitsfilter:NumberOfHitsFilter 0.000335432 0.00234659 0.0526657 0.000576179 0.00934575 30
produce:reco3d:SpacePointSolver 0.357983 1.99035 19.6057 0.850137 3.4528 30
produce:hitpdune:DisambigFromSpacePoints 0.306985 0.939925 2.6994 0.887965 0.516752 30
produce:pandora:StandardPandora 7.78567 21.5487 63.973 18.6473 11.8643 30
produce:pandoraWriter:StandardPandora 0.117123 0.250086 0.51596 0.241432 0.0767747 30
produce:pandoraTrack:LArPandoraTrackCreation 0.954583 3.31853 8.06522 3.17336 1.55266 30
produce:pandoraShower:LArPandoraShowerCreation 0.840067 4.31984 13.0624 4.36347 2.41593 30
produce:pandoracalo:Calorimetry 0.397324 1.43542 3.64469 1.36136 0.662842 30
produce:pandoracalonosce:Calorimetry 0.34112 1.34853 3.66728 1.28705 0.660132 30
produce:pandorapid:Chi2ParticleID 0.00147793 0.00307634 0.00567277 0.00296068 0.000824045 30
produce:pandoraShowercalo:ShowerCalorimetry 0.62939 3.63709 11.8184 3.47524 2.23309 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.844088 3.43406 11.6798 3.35426 2.17066 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0983308 0.213187 0.365677 0.220227 0.0620779 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0434007 0.0877221 0.147683 0.0886825 0.0253417 30
produce:emtrkmichelid:EmTrackMichelId 24.9518 44.2705 63.1565 44.6525 9.63742 30
[art]:TriggerResults:TriggerResultInserter 6.3607e-05 8.28234e-05 0.00012541 8.06465e-05 1.19281e-05 30
end_path:out1:RootOutput 1.6184e-05 2.35193e-05 3.9999e-05 2.36725e-05 5.166e-06 30
end_path:michelremoving:michelremoving 0.0349745 0.130585 0.741561 0.0754267 0.146426 30
end_path:michelremovinggnocchi:michelremoving 0.0339489 0.127052 0.715447 0.0728213 0.142984 30
end_path:calibanagnocchi:CalibAnaTree 0.472002 1.15413 3.04979 1.04477 0.580574 30
end_path:out1:RootOutput(write) 0.809861 1.30818 2.35821 1.25585 0.330497 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4976.53 MB
Peak resident set size usage (VmHWM): 2620.17 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0679_dataflow0_datawriter_0_20240905T035041_reco_stage1_reco_stage2_20240926T141347_keepup.root
\tHists: np04hd_raw_run029081_0679_dataflow0_datawriter_0_20240905T035041_reco_stage1_reco_stage2_20240926T141347_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
