Jobsub ID 263757.100@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263757.100@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:33 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce3 |
| Last heartbeat | 2024-09-26 15:04:30 |
| From worker node | Hostname | n221.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:20 |
| Input files | hd-protodune:np04hd_raw_run029092_0929_dataflow4_datawriter_0_20240906T075405.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:04:30 |
| Saved logs | justin-logs:263757.100-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
--------------------
Begin processing the 29th record. run: 29092 subRun: 1 event: 223188 at 26-Sep-2024 16:00:29 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 85692 traces, 85692 summary
Retagger: tagging trace set: gauss with 62011 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 62011 traces tagged "gauss"
FrameSaver: q=1.29711e+06 n=1100790 tag=gauss
wclsFrameSaver: saving 85692 traces tagged "wiener"
FrameSaver: q=1.4294e+06 n=1003557 tag=wiener
4951 hits in TPC 0
4066 hits in TPC 1
10526 hits in TPC 2
3854 hits in TPC 3
2580 hits in TPC 4
5006 hits in TPC 5
9540 hits in TPC 6
4664 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2916 XUs and 1438 XVs -> 6 XUVs
C:0 T:2 49253 XUs and 80332 XVs -> 5022 XUVs
C:0 T:3 3435 XUs and 6710 XVs -> 319 XUVs
C:0 T:4 149 XUs and 230 XVs -> 11 XUVs
C:0 T:5 8208 XUs and 8894 XVs -> 500 XUVs
C:0 T:6 59758 XUs and 72963 XVs -> 2282 XUVs
C:0 T:7 129 XUs and 166 XVs -> 8 XUVs
8148 XUVs total
2132 collection wire objects
8148 potential space points
Neighbour search...
597728 tests to find 343628 neighbours
Iterating with no regularization...
Begin: 1.2175e+07
0 1.06049e+07
1 1.04968e+07
2 1.04831e+07
3 1.04789e+07
Now with regularization...
Begin: 9.49844e+06
0 9.4933e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 4 event 223188 TrkackID 16 startx, y, z 14.6255 596.888 106.615 wireno, peakt tpcno 221 1350 2 dist 276.101min T, max_T6.13179 1387.27
no of hits closeby 4 event 223188 TrkackID 16 startx, y, z 14.6255 596.888 106.615 wireno, peakt tpcno 221 1350 2 dist 276.101min T, max_T6.13179 1387.27
Begin processing the 30th record. run: 29092 subRun: 1 event: 223196 at 26-Sep-2024 16:02:23 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 72834 traces, 72834 summary
Retagger: tagging trace set: gauss with 54101 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54101 traces tagged "gauss"
FrameSaver: q=1.16099e+06 n=1013456 tag=gauss
wclsFrameSaver: saving 72834 traces tagged "wiener"
FrameSaver: q=1.24739e+06 n=920941 tag=wiener
4718 hits in TPC 0
4664 hits in TPC 1
5056 hits in TPC 2
3638 hits in TPC 3
2601 hits in TPC 4
4687 hits in TPC 5
7961 hits in TPC 6
5035 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 381 XUs and 330 XVs -> 27 XUVs
C:0 T:2 13114 XUs and 14443 XVs -> 503 XUVs
C:0 T:3 33 XUs and 55 XVs -> 0 XUVs
C:0 T:4 77 XUs and 98 XVs -> 9 XUVs
C:0 T:5 20539 XUs and 32028 XVs -> 1957 XUVs
C:0 T:6 32328 XUs and 28889 XVs -> 1916 XUVs
C:0 T:7 820 XUs and 800 XVs -> 21 XUVs
4433 XUVs total
1428 collection wire objects
4433 potential space points
Neighbour search...
408863 tests to find 190156 neighbours
Iterating with no regularization...
Begin: 6.05457e+06
0 5.55354e+06
1 5.506e+06
2 5.49698e+06
3 5.49363e+06
Now with regularization...
Begin: 5.08215e+06
0 5.07958e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 223196 TrkackID 6 startx, y, z 75.638 195.984 28.5205 wireno, peakt tpcno 222 922.356 2 dist 138.209min T, max_T4.56109 950.391
no of hits closeby 0 event 223196 TrkackID 6 startx, y, z 75.638 195.984 28.5205 wireno, peakt tpcno 222 922.356 2 dist 138.209min T, max_T4.56109 950.391
26-Sep-2024 16:04:05 BST Closed output file "np04hd_raw_run029092_0929_dataflow4_datawriter_0_20240906T075405_reco_stage1_reco_stage2_20240926T150405_keepup.root"
26-Sep-2024 16:04:05 BST Closed input file "np04hd_raw_run029092_0929_dataflow4_datawriter_0_20240906T075405_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 78.977 104.669 125.384 103.592 11.8852 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000394302 0.00162707 0.00439657 0.00136088 0.00112066 30
produce:fembfilter:PDHDFEMBFilter 0.731837 0.866631 1.27542 0.759178 0.177829 30
produce:wclsdatahd:WireCellToolkit 44.636 52.8322 61.4209 54.262 4.68266 30
produce:gaushit:GausHitFinder 0.520154 0.83098 1.12268 0.839328 0.162663 30
produce:nhitsfilter:NumberOfHitsFilter 0.000265372 0.000395215 0.000763956 0.000388016 9.11794e-05 30
produce:reco3d:SpacePointSolver 0.200987 1.24895 3.33398 0.85181 0.830842 30
produce:hitpdune:DisambigFromSpacePoints 0.227116 0.625068 1.09047 0.581951 0.223136 30
produce:pandora:StandardPandora 6.95827 13.1202 23.9286 12.706 3.74781 30
produce:pandoraWriter:StandardPandora 0.129296 0.173589 0.225653 0.171191 0.0254842 30
produce:pandoraTrack:LArPandoraTrackCreation 1.25863 2.35359 4.0138 2.29072 0.692601 30
produce:pandoraShower:LArPandoraShowerCreation 1.33987 2.53482 4.71015 2.37035 0.88072 30
produce:pandoracalo:Calorimetry 0.639643 0.988142 1.63607 0.933023 0.24585 30
produce:pandoracalonosce:Calorimetry 0.601549 0.922181 1.56273 0.89555 0.232812 30
produce:pandorapid:Chi2ParticleID 0.00169591 0.00215141 0.00508705 0.0019713 0.00062141 30
produce:pandoraShowercalo:ShowerCalorimetry 1.06569 2.24914 3.82462 2.20642 0.702061 30
produce:pandoraShowercalonosce:ShowerCalorimetry 1.051 2.23902 4.40517 2.23979 0.749824 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0925461 0.146241 0.232854 0.150676 0.0304523 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0398654 0.0579904 0.0847885 0.0584154 0.0106486 30
produce:emtrkmichelid:EmTrackMichelId 16.0175 21.2613 29.2757 20.907 3.09536 30
[art]:TriggerResults:TriggerResultInserter 2.7594e-05 4.36642e-05 0.000122561 3.9102e-05 1.6905e-05 30
end_path:out1:RootOutput 5.984e-06 1.07034e-05 3.0691e-05 8.444e-06 5.18707e-06 30
end_path:michelremoving:michelremoving 0.0281797 0.108499 0.537815 0.06929 0.101363 30
end_path:michelremovinggnocchi:michelremoving 0.0266139 0.10714 0.534266 0.0688581 0.101467 30
end_path:calibanagnocchi:CalibAnaTree 0.405506 0.786231 1.56422 0.709988 0.264453 30
end_path:out1:RootOutput(write) 0.569947 1.01423 1.73545 0.965869 0.295974 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5804.3 MB
Peak resident set size usage (VmHWM): 2857.33 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0929_dataflow4_datawriter_0_20240906T075405_reco_stage1_reco_stage2_20240926T150405_keepup.root
\tHists: np04hd_raw_run029092_0929_dataflow4_datawriter_0_20240906T075405_reco_stage1_reco_stage2_20240926T150405_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
