Jobsub ID 263757.112@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263757.112@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:33 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:48:11 |
| From worker node | Hostname | n255.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:14 |
| Input files | hd-protodune:np04hd_raw_run029092_0925_dataflow1_datawriter_0_20240906T074917.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:48:11 |
| Saved logs | justin-logs:263757.112-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
gularization...
Begin: 3.51037e+07
0 2.96763e+07
1 2.88808e+07
2 2.88398e+07
3 2.88346e+07
Now with regularization...
Begin: 2.7791e+07
0 2.77801e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 222217 TrkackID 14 startx, y, z 61.3673 175.627 452.308 wireno, peakt tpcno 310 5025.92 5 dist 0min T, max_T4602.07 5365.21
no of hits closeby 0 event 222217 TrkackID 14 startx, y, z 61.3673 175.627 452.308 wireno, peakt tpcno 310 5025.92 5 dist 0min T, max_T4602.07 5365.21
Begin processing the 29th record. run: 29092 subRun: 1 event: 222233 at 26-Sep-2024 15:44:55 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 94702 traces, 94702 summary
Retagger: tagging trace set: gauss with 68051 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 68051 traces tagged "gauss"
FrameSaver: q=1.50114e+06 n=1213684 tag=gauss
wclsFrameSaver: saving 94702 traces tagged "wiener"
FrameSaver: q=1.662e+06 n=1092421 tag=wiener
5172 hits in TPC 0
6176 hits in TPC 1
8106 hits in TPC 2
3663 hits in TPC 3
4981 hits in TPC 4
10905 hits in TPC 5
7774 hits in TPC 6
2846 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 5236 XUs and 2251 XVs -> 405 XUVs
C:0 T:2 50030 XUs and 48223 XVs -> 3599 XUVs
C:0 T:3 2996 XUs and 1646 XVs -> 83 XUVs
C:0 T:4 23 XUs and 29 XVs -> 1 XUVs
C:0 T:5 82099 XUs and 98452 XVs -> 5018 XUVs
C:0 T:6 47809 XUs and 52143 XVs -> 750 XUVs
C:0 T:7 28536 XUs and 21007 XVs -> 2241 XUVs
12097 XUVs total
3150 collection wire objects
12097 potential space points
Neighbour search...
820229 tests to find 406660 neighbours
Iterating with no regularization...
Begin: 1.93423e+07
0 1.71285e+07
1 1.69578e+07
2 1.69275e+07
3 1.69162e+07
Now with regularization...
Begin: 1.58675e+07
0 1.58608e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 222233 TrkackID 8 startx, y, z 78.8023 592.202 347.386 wireno, peakt tpcno 93 1790.88 6 dist 299.77min T, max_T6.73997 1806.47
no of hits closeby 0 event 222233 TrkackID 8 startx, y, z 78.8023 592.202 347.386 wireno, peakt tpcno 93 1790.88 6 dist 299.77min T, max_T6.73997 1806.47
Begin processing the 30th record. run: 29092 subRun: 1 event: 222241 at 26-Sep-2024 15:46:53 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 34760 traces, 34760 summary
Retagger: tagging trace set: gauss with 26438 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 26438 traces tagged "gauss"
FrameSaver: q=549191 n=476264 tag=gauss
wclsFrameSaver: saving 34760 traces tagged "wiener"
FrameSaver: q=590549 n=430560 tag=wiener
1918 hits in TPC 0
2048 hits in TPC 1
3537 hits in TPC 2
2745 hits in TPC 3
1157 hits in TPC 4
2184 hits in TPC 5
3366 hits in TPC 6
1444 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2079 XUs and 849 XVs -> 23 XUVs
C:0 T:2 17581 XUs and 13355 XVs -> 382 XUVs
C:0 T:3 572 XUs and 835 XVs -> 35 XUVs
C:0 T:4 1 XUs and 5 XVs -> 0 XUVs
C:0 T:5 1323 XUs and 2554 XVs -> 245 XUVs
C:0 T:6 4457 XUs and 5317 XVs -> 719 XUVs
C:0 T:7 244 XUs and 469 XVs -> 12 XUVs
1416 XUVs total
722 collection wire objects
1416 potential space points
Neighbour search...
32176 tests to find 19742 neighbours
Iterating with no regularization...
Begin: 2.76806e+06
0 2.49651e+06
1 2.46641e+06
2 2.46231e+06
3 2.46133e+06
Now with regularization...
Begin: 2.1917e+06
0 2.19113e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:47:36 BST Closed output file "np04hd_raw_run029092_0925_dataflow1_datawriter_0_20240906T074917_reco_stage1_reco_stage2_20240926T144736_keepup.root"
26-Sep-2024 15:47:36 BST Closed input file "np04hd_raw_run029092_0925_dataflow1_datawriter_0_20240906T074917_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 42.6855 76.9088 117.085 73.7293 18.1013 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000553269 0.00194906 0.00416495 0.00178884 0.000926612 30
produce:fembfilter:PDHDFEMBFilter 0.451612 0.606667 0.693663 0.636675 0.084273 30
produce:wclsdatahd:WireCellToolkit 25.3243 32.1377 36.8002 34.1576 4.0196 30
produce:gaushit:GausHitFinder 0.241837 0.634423 1.13081 0.564719 0.220111 30
produce:nhitsfilter:NumberOfHitsFilter 0.000200891 0.000327522 0.000634994 0.000298711 9.8137e-05 30
produce:reco3d:SpacePointSolver 0.132458 1.02635 4.653 0.626166 1.01342 30
produce:hitpdune:DisambigFromSpacePoints 0.0973867 0.511152 1.38106 0.364956 0.33374 30
produce:pandora:StandardPandora 2.46506 11.8394 31.9531 9.7014 6.79814 30
produce:pandoraWriter:StandardPandora 0.0559985 0.144632 0.222581 0.140803 0.0421641 30
produce:pandoraTrack:LArPandoraTrackCreation 0.281858 1.08817 2.37796 0.967273 0.54608 30
produce:pandoraShower:LArPandoraShowerCreation 0.225958 1.2895 3.40529 1.17737 0.792735 30
produce:pandoracalo:Calorimetry 0.150384 0.56234 1.15385 0.513725 0.241341 30
produce:pandoracalonosce:Calorimetry 0.129796 0.502899 1.04653 0.477034 0.223899 30
produce:pandorapid:Chi2ParticleID 0.000909324 0.00177362 0.00283132 0.00173607 0.000471018 30
produce:pandoraShowercalo:ShowerCalorimetry 0.193883 1.15506 3.45927 0.987616 0.73772 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.191649 1.12271 3.46046 0.962344 0.72198 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0455043 0.130834 0.232603 0.120019 0.0425074 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0161717 0.0434316 0.0740603 0.0381903 0.0133336 30
produce:emtrkmichelid:EmTrackMichelId 10.1267 22.6749 36.1662 22.6298 6.40216 30
[art]:TriggerResults:TriggerResultInserter 1.4998e-05 3.28971e-05 8.8347e-05 3.07935e-05 1.39562e-05 30
end_path:out1:RootOutput 7.915e-06 1.84526e-05 0.000213145 1.1437e-05 3.62989e-05 30
end_path:michelremoving:michelremoving 0.0155721 0.0755649 0.227738 0.0323792 0.0678405 30
end_path:michelremovinggnocchi:michelremoving 0.0146502 0.0727812 0.222901 0.0271226 0.0671831 30
end_path:calibanagnocchi:CalibAnaTree 0.170568 0.421818 0.945334 0.35954 0.186559 30
end_path:out1:RootOutput(write) 0.432299 0.736227 1.19154 0.661404 0.213338 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4913.35 MB
Peak resident set size usage (VmHWM): 2587.04 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0925_dataflow1_datawriter_0_20240906T074917_reco_stage1_reco_stage2_20240926T144736_keepup.root
\tHists: np04hd_raw_run029092_0925_dataflow1_datawriter_0_20240906T074917_reco_stage1_reco_stage2_20240926T144736_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
