Jobsub ID 263765.79@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263765.79@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:52 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 14:45:07 |
| From worker node | Hostname | wn029.hep |
| cpuinfo | Intel(R) Xeon(R) Silver 4216 CPU @ 2.10GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:26:28 |
| Input files | hd-protodune:np04hd_raw_run029107_2219_dataflow4_datawriter_0_20240908T035240.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:45:07 |
| Saved logs | justin-logs:263765.79-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
643 wireno, peakt tpcno 38 5.52662 2 dist 0min T, max_T5.52662 2481
no of hits closeby 0 event 532804 TrkackID 3 startx, y, z 199.556 604.431 144.824 wireno, peakt tpcno 353 4004.5 2 dist 449.129min T, max_T8.59579 4022.61
no of hits closeby 0 event 532804 TrkackID 25 startx, y, z 43.4981 600.74 12.0508 wireno, peakt tpcno 67 988.814 2 dist 183.008min T, max_T8.07885 1127.09
no of hits closeby 0 event 532804 TrkackID 2 startx, y, z -191.967 424.489 337.643 wireno, peakt tpcno 38 5.52662 2 dist 0min T, max_T5.52662 2481
no of hits closeby 0 event 532804 TrkackID 3 startx, y, z 199.556 604.431 144.824 wireno, peakt tpcno 353 4004.5 2 dist 449.129min T, max_T8.59579 4022.61
no of hits closeby 0 event 532804 TrkackID 25 startx, y, z 43.4981 600.74 12.0508 wireno, peakt tpcno 67 988.814 2 dist 183.008min T, max_T8.07885 1127.09
Begin processing the 29th record. run: 29107 subRun: 1 event: 532828 at 26-Sep-2024 15:41:16 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 77582 traces, 77582 summary
Retagger: tagging trace set: gauss with 57513 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 57513 traces tagged "gauss"
FrameSaver: q=1.24343e+06 n=1105861 tag=gauss
wclsFrameSaver: saving 77582 traces tagged "wiener"
FrameSaver: q=1.32987e+06 n=992643 tag=wiener
5571 hits in TPC 0
4023 hits in TPC 1
10164 hits in TPC 2
6256 hits in TPC 3
2036 hits in TPC 4
2595 hits in TPC 5
6446 hits in TPC 6
3971 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 19 XUs and 16 XVs -> 0 XUVs
C:0 T:1 2531 XUs and 2288 XVs -> 142 XUVs
C:0 T:2 45999 XUs and 54395 XVs -> 1793 XUVs
C:0 T:3 174 XUs and 242 XVs -> 5 XUVs
C:0 T:4 59 XUs and 45 XVs -> 1 XUVs
C:0 T:5 10378 XUs and 8011 XVs -> 160 XUVs
C:0 T:6 33922 XUs and 27149 XVs -> 768 XUVs
C:0 T:7 30 XUs and 60 XVs -> 1 XUVs
2870 XUVs total
1649 collection wire objects
2870 potential space points
Neighbour search...
52598 tests to find 28902 neighbours
Iterating with no regularization...
Begin: 5.1669e+06
0 4.83048e+06
1 4.81122e+06
2 4.80931e+06
Now with regularization...
Begin: 4.40553e+06
0 4.40493e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 532828 TrkackID 7 startx, y, z 79.0556 595.871 187.589 wireno, peakt tpcno 334 3621.86 2 dist 463.437min T, max_T9.16007 3702.33
no of hits closeby 0 event 532828 TrkackID 7 startx, y, z 79.0556 595.871 187.589 wireno, peakt tpcno 334 3621.86 2 dist 463.437min T, max_T9.16007 3702.33
Begin processing the 30th record. run: 29107 subRun: 1 event: 532836 at 26-Sep-2024 15:43:08 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58328 traces, 58328 summary
Retagger: tagging trace set: gauss with 45776 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 45776 traces tagged "gauss"
FrameSaver: q=1.02438e+06 n=924348 tag=gauss
wclsFrameSaver: saving 58328 traces tagged "wiener"
FrameSaver: q=1.07376e+06 n=845858 tag=wiener
2338 hits in TPC 0
2993 hits in TPC 1
5769 hits in TPC 2
3626 hits in TPC 3
2773 hits in TPC 4
3941 hits in TPC 5
5803 hits in TPC 6
4033 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 315 XUs and 875 XVs -> 67 XUVs
C:0 T:2 13615 XUs and 13426 XVs -> 322 XUVs
C:0 T:3 205 XUs and 242 XVs -> 13 XUVs
C:0 T:4 35 XUs and 70 XVs -> 6 XUVs
C:0 T:5 6900 XUs and 10202 XVs -> 906 XUVs
C:0 T:6 15358 XUs and 11420 XVs -> 580 XUVs
C:0 T:7 170 XUs and 124 XVs -> 2 XUVs
1896 XUVs total
1065 collection wire objects
1896 potential space points
Neighbour search...
32632 tests to find 19124 neighbours
Iterating with no regularization...
Begin: 4.17889e+06
0 3.97395e+06
1 3.95519e+06
2 3.95253e+06
Now with regularization...
Begin: 3.55323e+06
0 3.55249e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:44:44 BST Closed output file "np04hd_raw_run029107_2219_dataflow4_datawriter_0_20240908T035240_reco_stage1_reco_stage2_20240926T144444_keepup.root"
26-Sep-2024 15:44:44 BST Closed input file "np04hd_raw_run029107_2219_dataflow4_datawriter_0_20240908T035240_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 74.2275 96.8553 121.446 96.5547 13.5689 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000371936 0.00127269 0.00339211 0.000830973 0.000974198 30
produce:fembfilter:PDHDFEMBFilter 0.698912 0.714889 0.765324 0.712852 0.010702 30
produce:wclsdatahd:WireCellToolkit 44.3486 46.9758 52.4777 46.6904 1.7166 30
produce:gaushit:GausHitFinder 0.496062 0.782575 1.10459 0.819279 0.16579 30
produce:nhitsfilter:NumberOfHitsFilter 0.000286825 0.000379357 0.000471481 0.000387772 5.66056e-05 30
produce:reco3d:SpacePointSolver 0.199463 1.00042 3.09256 0.790692 0.704081 30
produce:hitpdune:DisambigFromSpacePoints 0.237701 0.548768 0.920276 0.556577 0.194188 30
produce:pandora:StandardPandora 6.58877 13.1497 35.3433 12.671 5.88846 30
produce:pandoraWriter:StandardPandora 0.0944922 0.154647 0.200951 0.161575 0.0308592 30
produce:pandoraTrack:LArPandoraTrackCreation 0.623329 1.51529 2.70245 1.44459 0.551166 30
produce:pandoraShower:LArPandoraShowerCreation 0.627606 1.69693 4.39755 1.65578 0.818521 30
produce:pandoracalo:Calorimetry 0.305399 0.6737 1.07489 0.675638 0.214118 30
produce:pandoracalonosce:Calorimetry 0.277268 0.611657 0.99921 0.619904 0.195924 30
produce:pandorapid:Chi2ParticleID 0.00126355 0.00331782 0.0474956 0.00175407 0.00820984 30
produce:pandoraShowercalo:ShowerCalorimetry 0.500301 1.4757 3.21194 1.49567 0.667049 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.493393 1.44662 3.16066 1.49226 0.662667 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0787516 0.127774 0.185034 0.128165 0.0276173 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0311572 0.0490476 0.0708339 0.0495294 0.0103691 30
produce:emtrkmichelid:EmTrackMichelId 17.1949 24.2954 31.1635 24.5246 4.27995 30
[art]:TriggerResults:TriggerResultInserter 2.5503e-05 4.03715e-05 7.8696e-05 3.77455e-05 9.232e-06 30
end_path:out1:RootOutput 8.253e-06 1.22038e-05 2.6069e-05 1.0893e-05 3.4571e-06 30
end_path:michelremoving:michelremoving 0.0175722 0.09128 0.435249 0.0435243 0.0922371 30
end_path:michelremovinggnocchi:michelremoving 0.0145933 0.0876913 0.437469 0.0395615 0.093626 30
end_path:calibanagnocchi:CalibAnaTree 0.210772 0.496012 0.831992 0.486413 0.175503 30
end_path:out1:RootOutput(write) 0.547659 0.785204 1.06613 0.796171 0.153751 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5760.94 MB
Peak resident set size usage (VmHWM): 2595.02 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2219_dataflow4_datawriter_0_20240908T035240_reco_stage1_reco_stage2_20240926T144444_keepup.root
\tHists: np04hd_raw_run029107_2219_dataflow4_datawriter_0_20240908T035240_reco_stage1_reco_stage2_20240926T144444_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
