Jobsub ID 263933.0@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263933.0@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:23:59 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 15:54:31 |
| From worker node | Hostname | wn096.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:25:41 |
| Input files | hd-protodune:np04hd_raw_run029107_2260_dataflow4_datawriter_0_20240908T042958.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:54:31 |
| Saved logs | justin-logs:263933.0-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
119 XUVs
C:0 T:3 605 XUs and 1178 XVs -> 47 XUVs
C:0 T:4 81 XUs and 256 XVs -> 19 XUVs
C:0 T:5 16694 XUs and 23194 XVs -> 5052 XUVs
C:0 T:6 15404 XUs and 6761 XVs -> 400 XUVs
C:0 T:7 61 XUs and 28 XVs -> 2 XUVs
6557 XUVs total
697 collection wire objects
6557 potential space points
Neighbour search...
1247401 tests to find 598364 neighbours
Iterating with no regularization...
Begin: 4.36765e+06
0 3.86908e+06
1 3.83113e+06
2 3.82519e+06
3 3.82367e+06
Now with regularization...
Begin: 3.58322e+06
0 3.58092e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 542644 TrkackID 3 startx, y, z 36.0922 595.766 125.721 wireno, peakt tpcno 255 3753.58 2 dist 381.272min T, max_T11.6239 3760.5
no of hits closeby 0 event 542644 TrkackID 3 startx, y, z 36.0922 595.766 125.721 wireno, peakt tpcno 255 3753.58 2 dist 381.272min T, max_T11.6239 3760.5
Begin processing the 29th record. run: 29107 subRun: 1 event: 542652 at 26-Sep-2024 16:49:19 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 54924 traces, 54924 summary
Retagger: tagging trace set: gauss with 42850 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 42850 traces tagged "gauss"
FrameSaver: q=941936 n=841530 tag=gauss
wclsFrameSaver: saving 54924 traces tagged "wiener"
FrameSaver: q=997063 n=766060 tag=wiener
2574 hits in TPC 0
2709 hits in TPC 1
6547 hits in TPC 2
3564 hits in TPC 3
2954 hits in TPC 4
4006 hits in TPC 5
5549 hits in TPC 6
2121 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 0 XUs and 0 XVs -> 0 XUVs
C:0 T:1 747 XUs and 998 XVs -> 11 XUVs
C:0 T:2 22242 XUs and 27348 XVs -> 714 XUVs
C:0 T:3 198 XUs and 299 XVs -> 6 XUVs
C:0 T:4 275 XUs and 387 XVs -> 46 XUVs
C:0 T:5 7088 XUs and 6456 XVs -> 349 XUVs
C:0 T:6 13237 XUs and 12739 XVs -> 328 XUVs
C:0 T:7 222 XUs and 149 XVs -> 4 XUVs
1458 XUVs total
1042 collection wire objects
1458 potential space points
Neighbour search...
18364 tests to find 10948 neighbours
Iterating with no regularization...
Begin: 7.6895e+06
0 7.39685e+06
1 7.38311e+06
2 7.3809e+06
Now with regularization...
Begin: 6.88938e+06
0 6.88842e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 542660 at 26-Sep-2024 16:51:10 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 103718 traces, 103718 summary
Retagger: tagging trace set: gauss with 75333 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 75333 traces tagged "gauss"
FrameSaver: q=1.61677e+06 n=1359746 tag=gauss
wclsFrameSaver: saving 103718 traces tagged "wiener"
FrameSaver: q=1.78523e+06 n=1230328 tag=wiener
4003 hits in TPC 0
4637 hits in TPC 1
6553 hits in TPC 2
5073 hits in TPC 3
4319 hits in TPC 4
8667 hits in TPC 5
12359 hits in TPC 6
9114 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 8519 XUs and 11348 XVs -> 1362 XUVs
C:0 T:2 39255 XUs and 80419 XVs -> 4380 XUVs
C:0 T:3 130 XUs and 136 XVs -> 9 XUVs
C:0 T:4 76 XUs and 91 XVs -> 7 XUVs
C:0 T:5 51063 XUs and 93608 XVs -> 1567 XUVs
C:0 T:6 93388 XUs and 71488 XVs -> 2137 XUVs
C:0 T:7 1266 XUs and 1353 XVs -> 78 XUVs
9540 XUVs total
2758 collection wire objects
9540 potential space points
Neighbour search...
987420 tests to find 482198 neighbours
Iterating with no regularization...
Begin: 1.46669e+07
0 1.31095e+07
1 1.29939e+07
2 1.29768e+07
3 1.29713e+07
Now with regularization...
Begin: 1.2321e+07
0 1.23173e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:54:09 BST Closed output file "np04hd_raw_run029107_2260_dataflow4_datawriter_0_20240908T042958_reco_stage1_reco_stage2_20240926T155409_keepup.root"
26-Sep-2024 16:54:09 BST Closed input file "np04hd_raw_run029107_2260_dataflow4_datawriter_0_20240908T042958_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 56.7696 117.875 178.045 118.873 24.0728 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000460296 0.0029241 0.00468681 0.0032659 0.00108835 30
produce:fembfilter:PDHDFEMBFilter 0.669415 0.753805 0.922444 0.75682 0.0580393 30
produce:wclsdatahd:WireCellToolkit 52.7777 56.3279 61.2274 55.8607 1.90723 30
produce:gaushit:GausHitFinder 0.423112 0.802178 1.34499 0.845929 0.235682 30
produce:nhitsfilter:NumberOfHitsFilter 0.000203097 0.000414468 0.000634033 0.000442656 0.000105868 30
produce:reco3d:SpacePointSolver 0.143049 1.03675 4.03273 0.853812 0.902371 29
produce:hitpdune:DisambigFromSpacePoints 0.120259 0.621636 1.86028 0.619081 0.365794 29
produce:pandora:StandardPandora 5.37909 15.3007 33.6251 15.3334 6.65034 29
produce:pandoraWriter:StandardPandora 0.0963365 0.172432 0.288736 0.168781 0.0406195 29
produce:pandoraTrack:LArPandoraTrackCreation 0.650797 1.65997 4.72865 1.38082 0.880553 29
produce:pandoraShower:LArPandoraShowerCreation 0.497944 2.00256 5.56525 1.72623 1.23656 29
produce:pandoracalo:Calorimetry 0.341433 0.849297 1.96968 0.778597 0.396973 29
produce:pandoracalonosce:Calorimetry 0.298456 0.789201 1.76768 0.739853 0.362272 29
produce:pandorapid:Chi2ParticleID 0.00147061 0.00214097 0.00364375 0.00208978 0.000468957 29
produce:pandoraShowercalo:ShowerCalorimetry 0.458384 1.90067 5.51454 1.62448 1.21267 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.440156 1.83928 5.34893 1.57249 1.16868 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0821081 0.143032 0.228436 0.137356 0.0368267 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0317799 0.0515638 0.0816897 0.0485818 0.0127291 29
produce:emtrkmichelid:EmTrackMichelId 19.1541 33.9084 57.6404 33.7976 8.22318 29
[art]:TriggerResults:TriggerResultInserter 2.8137e-05 4.54822e-05 0.000115617 4.1506e-05 1.60737e-05 30
end_path:out1:RootOutput 9.198e-06 1.20928e-05 3.1238e-05 1.04105e-05 5.20764e-06 30
end_path:michelremoving:michelremoving 3.4053e-05 0.0957206 0.320315 0.0575918 0.0744729 30
end_path:michelremovinggnocchi:michelremoving 1.2003e-05 0.0862403 0.313944 0.0513259 0.0705665 30
end_path:calibanagnocchi:CalibAnaTree 1.3497e-05 0.601938 1.38372 0.527634 0.298715 30
end_path:out1:RootOutput(write) 0.335383 0.812306 1.34407 0.820853 0.232142 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4950.51 MB
Peak resident set size usage (VmHWM): 2349.23 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2260_dataflow4_datawriter_0_20240908T042958_reco_stage1_reco_stage2_20240926T155409_keepup.root
\tHists: np04hd_raw_run029107_2260_dataflow4_datawriter_0_20240908T042958_reco_stage1_reco_stage2_20240926T155409_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
