Jobsub ID 79157.0@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 79157.0@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3460 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:49:40 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce01 |
| Last heartbeat | 2024-09-26 15:51:58 |
| From worker node | Hostname | wn2209240.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7513 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:50:45 |
| Input files | hd-protodune:np04hd_raw_run029218_0090_dataflow5_datawriter_0_20240917T164733.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:51:58 |
| Saved logs | justin-logs:79157.0-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
mParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 22557 TrkackID 12 startx, y, z 102.897 519.026 1.77983 wireno, peakt tpcno 71 1390.49 2 dist 158.278min T, max_T13.6096 1419.51
no of hits closeby 0 event 22557 TrkackID 12 startx, y, z 102.897 519.026 1.77983 wireno, peakt tpcno 71 1390.49 2 dist 158.278min T, max_T13.6096 1419.51
Begin processing the 30th record. run: 29218 subRun: 1 event: 22565 at 26-Sep-2024 16:49:28 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 48141 traces, 48141 summary
Retagger: tagging trace set: gauss with 35357 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 35357 traces tagged "gauss"
FrameSaver: q=782700 n=630673 tag=gauss
wclsFrameSaver: saving 48141 traces tagged "wiener"
FrameSaver: q=847959 n=566902 tag=wiener
649 hits in TPC 0
209 hits in TPC 1
5662 hits in TPC 2
2748 hits in TPC 3
3145 hits in TPC 4
3824 hits in TPC 5
4675 hits in TPC 6
3579 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:2 17558 XUs and 20334 XVs -> 651 XUVs
C:0 T:3 245 XUs and 308 XVs -> 6 XUVs
C:0 T:4 161 XUs and 132 XVs -> 20 XUVs
C:0 T:5 40011 XUs and 25109 XVs -> 1378 XUVs
C:0 T:6 20278 XUs and 21508 XVs -> 1094 XUVs
C:0 T:7 0 XUs and 0 XVs -> 0 XUVs
3149 XUVs total
1282 collection wire objects
3149 potential space points
Neighbour search...
104167 tests to find 62484 neighbours
Iterating with no regularization...
Begin: 1.41871e+07
0 1.24737e+07
1 1.24024e+07
2 1.23885e+07
3 1.23832e+07
Now with regularization...
Begin: 1.17842e+07
0 1.1781e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 7 event 22565 TrkackID 4 startx, y, z 28.5475 504.883 8.60618 wireno, peakt tpcno 77 5854.8 2 dist 443.72min T, max_T3018.93 5855.49
no of hits closeby 7 event 22565 TrkackID 4 startx, y, z 28.5475 504.883 8.60618 wireno, peakt tpcno 77 5854.8 2 dist 443.72min T, max_T3018.93 5855.49
Begin processing the 31st record. run: 29218 subRun: 1 event: 22573 at 26-Sep-2024 16:50:26 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 47130 traces, 47130 summary
Retagger: tagging trace set: gauss with 35262 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 35262 traces tagged "gauss"
FrameSaver: q=771118 n=690104 tag=gauss
wclsFrameSaver: saving 47130 traces tagged "wiener"
FrameSaver: q=839695 n=629446 tag=wiener
2206 hits in TPC 0
2727 hits in TPC 1
6049 hits in TPC 2
3123 hits in TPC 3
2343 hits in TPC 4
4225 hits in TPC 5
3186 hits in TPC 6
2617 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 184 XUs and 716 XVs -> 21 XUVs
C:0 T:2 18711 XUs and 36961 XVs -> 1219 XUVs
C:0 T:3 87 XUs and 184 XVs -> 4 XUVs
C:0 T:4 4 XUs and 1 XVs -> 0 XUVs
C:0 T:5 11752 XUs and 28230 XVs -> 1116 XUVs
C:0 T:6 9542 XUs and 36485 XVs -> 1589 XUVs
C:0 T:7 4 XUs and 1 XVs -> 0 XUVs
3949 XUVs total
1319 collection wire objects
3949 potential space points
Neighbour search...
245371 tests to find 122896 neighbours
Iterating with no regularization...
Begin: 8.26929e+06
0 6.8524e+06
1 6.73424e+06
2 6.72524e+06
3 6.72293e+06
Now with regularization...
Begin: 6.50436e+06
0 6.50324e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 22573 TrkackID 3 startx, y, z 242.292 284.958 178.09 wireno, peakt tpcno 74 2324.88 5 dist 0min T, max_T4.38269 3088.83
no of hits closeby 1 event 22573 TrkackID 3 startx, y, z 242.292 284.958 178.09 wireno, peakt tpcno 74 2324.88 5 dist 0min T, max_T4.38269 3088.83
26-Sep-2024 16:51:30 BST Closed output file "np04hd_raw_run029218_0090_dataflow5_datawriter_0_20240917T164733_reco_stage1_reco_stage2_20240926T155130_keepup.root"
26-Sep-2024 16:51:30 BST Closed input file "np04hd_raw_run029218_0090_dataflow5_datawriter_0_20240917T164733_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 36.6299 73.4187 107.989 72.6971 15.0325 31
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000410772 0.00251603 0.00452728 0.00266195 0.000954982 31
produce:fembfilter:PDHDFEMBFilter 0.479512 0.635059 0.812001 0.662739 0.118437 31
produce:wclsdatahd:WireCellToolkit 28.8516 33.2736 39.9322 32.0029 3.50263 31
produce:gaushit:GausHitFinder 0.346655 0.569935 1.12034 0.547524 0.164655 31
produce:nhitsfilter:NumberOfHitsFilter 0.000231075 0.000443855 0.00219508 0.000339307 0.000366116 31
produce:reco3d:SpacePointSolver 0.155476 0.546484 1.56855 0.508553 0.298797 30
produce:hitpdune:DisambigFromSpacePoints 0.143262 0.355185 0.708674 0.356112 0.146755 30
produce:pandora:StandardPandora 3.76569 9.38714 17.7761 8.25722 3.95044 30
produce:pandoraWriter:StandardPandora 0.0843757 0.159864 0.313604 0.144939 0.0574597 30
produce:pandoraTrack:LArPandoraTrackCreation 0.409723 0.932964 2.31302 0.747026 0.472026 30
produce:pandoraShower:LArPandoraShowerCreation 0.430278 1.05848 3.57316 0.869871 0.6459 30
produce:pandoracalo:Calorimetry 0.222355 0.492614 0.975713 0.439475 0.21158 30
produce:pandoracalonosce:Calorimetry 0.198188 0.438151 1.04522 0.36394 0.206083 30
produce:pandorapid:Chi2ParticleID 0.00100476 0.00179191 0.00390308 0.00164523 0.00064754 30
produce:pandoraShowercalo:ShowerCalorimetry 0.32397 0.96648 2.37232 0.793302 0.501542 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.324512 0.94123 2.39861 0.877333 0.490281 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0577266 0.11286 0.259608 0.105906 0.0421492 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0200109 0.0398262 0.0890751 0.0345959 0.0152177 30
produce:emtrkmichelid:EmTrackMichelId 15.1806 23.4432 38.7513 22.5448 5.47952 30
[art]:TriggerResults:TriggerResultInserter 1.8525e-05 4.1199e-05 0.000153689 3.5747e-05 2.40086e-05 31
end_path:out1:RootOutput 7.965e-06 1.46495e-05 5.0475e-05 1.2263e-05 8.3556e-06 31
end_path:michelremoving:michelremoving 4.9483e-05 0.0541001 0.192948 0.0292168 0.0537768 31
end_path:michelremovinggnocchi:michelremoving 3.5286e-05 0.0519706 0.205238 0.0232497 0.0566288 31
end_path:calibanagnocchi:CalibAnaTree 1.5029e-05 0.346552 0.823675 0.304633 0.16259 31
end_path:out1:RootOutput(write) 0.444458 0.735778 1.22283 0.769025 0.186356 31
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4937.06 MB
Peak resident set size usage (VmHWM): 2578.62 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029218_0090_dataflow5_datawriter_0_20240917T164733_reco_stage1_reco_stage2_20240926T155130_keepup.root
\tHists: np04hd_raw_run029218_0090_dataflow5_datawriter_0_20240917T164733_reco_stage1_reco_stage2_20240926T155130_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
