Jobsub ID 263952.31@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263952.31@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:28:43 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 15:59:53 |
| From worker node | Hostname | wn097.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:30:09 |
| Input files | hd-protodune:np04hd_raw_run029107_0766_dataflow4_datawriter_0_20240907T032948.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:59:53 |
| Saved logs | justin-logs:263952.31-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
T:3 160 XUs and 257 XVs -> 11 XUVs
C:0 T:4 156 XUs and 276 XVs -> 13 XUVs
C:0 T:5 38140 XUs and 58520 XVs -> 4454 XUVs
C:0 T:6 31023 XUs and 36070 XVs -> 1127 XUVs
C:0 T:7 67 XUs and 30 XVs -> 1 XUVs
7496 XUVs total
1799 collection wire objects
7496 potential space points
Neighbour search...
1009452 tests to find 476856 neighbours
Iterating with no regularization...
Begin: 6.19599e+06
0 5.28254e+06
1 5.22618e+06
2 5.21644e+06
3 5.21351e+06
Now with regularization...
Begin: 4.87925e+06
0 4.87757e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 184132 at 26-Sep-2024 16:55:23 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 57761 traces, 57761 summary
Retagger: tagging trace set: gauss with 43284 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43284 traces tagged "gauss"
FrameSaver: q=952746 n=804555 tag=gauss
wclsFrameSaver: saving 57761 traces tagged "wiener"
FrameSaver: q=1.01869e+06 n=728494 tag=wiener
3326 hits in TPC 0
2573 hits in TPC 1
6155 hits in TPC 2
3904 hits in TPC 3
2811 hits in TPC 4
4571 hits in TPC 5
4387 hits in TPC 6
2404 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1408 XUs and 1000 XVs -> 78 XUVs
C:0 T:2 25763 XUs and 39005 XVs -> 1981 XUVs
C:0 T:3 59 XUs and 108 XVs -> 1 XUVs
C:0 T:4 55 XUs and 39 XVs -> 0 XUVs
C:0 T:5 12019 XUs and 8329 XVs -> 359 XUVs
C:0 T:6 9626 XUs and 9189 XVs -> 145 XUVs
C:0 T:7 212 XUs and 98 XVs -> 1 XUVs
2565 XUVs total
1151 collection wire objects
2565 potential space points
Neighbour search...
140039 tests to find 69630 neighbours
Iterating with no regularization...
Begin: 7.21533e+06
0 6.74872e+06
1 6.72274e+06
2 6.71896e+06
Now with regularization...
Begin: 6.36961e+06
0 6.36848e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 184148 at 26-Sep-2024 16:57:17 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 64198 traces, 64198 summary
Retagger: tagging trace set: gauss with 46953 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 46953 traces tagged "gauss"
FrameSaver: q=1.04452e+06 n=865231 tag=gauss
wclsFrameSaver: saving 64198 traces tagged "wiener"
FrameSaver: q=1.1193e+06 n=781536 tag=wiener
4230 hits in TPC 0
2960 hits in TPC 1
5128 hits in TPC 2
1987 hits in TPC 3
2113 hits in TPC 4
3351 hits in TPC 5
6690 hits in TPC 6
5433 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 4 XUs and 2 XVs -> 0 XUVs
C:0 T:1 65 XUs and 67 XVs -> 3 XUVs
C:0 T:2 10142 XUs and 12031 XVs -> 396 XUVs
C:0 T:3 97 XUs and 159 XVs -> 6 XUVs
C:0 T:4 47 XUs and 91 XVs -> 2 XUVs
C:0 T:5 6179 XUs and 8232 XVs -> 537 XUVs
C:0 T:6 25498 XUs and 18656 XVs -> 1445 XUVs
C:0 T:7 468 XUs and 369 XVs -> 6 XUVs
2395 XUVs total
1436 collection wire objects
2395 potential space points
Neighbour search...
39271 tests to find 21794 neighbours
Iterating with no regularization...
Begin: 7.59615e+06
0 7.26613e+06
1 7.23947e+06
2 7.23753e+06
Now with regularization...
Begin: 6.44344e+06
0 6.44289e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:59:14 BST Closed output file "np04hd_raw_run029107_0766_dataflow4_datawriter_0_20240907T032948_reco_stage1_reco_stage2_20240926T155914_keepup.root"
26-Sep-2024 16:59:14 BST Closed input file "np04hd_raw_run029107_0766_dataflow4_datawriter_0_20240907T032948_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 70.3368 111.695 154.599 109.637 18.7538 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000442164 0.00215076 0.00372771 0.00200632 0.000946777 30
produce:fembfilter:PDHDFEMBFilter 0.670558 0.689313 0.710904 0.689522 0.0110427 30
produce:wclsdatahd:WireCellToolkit 44.7489 49.8119 61.9022 47.4061 5.01892 30
produce:gaushit:GausHitFinder 0.333346 0.828301 1.2804 0.804724 0.199783 30
produce:nhitsfilter:NumberOfHitsFilter 0.000192223 0.000359915 0.000554242 0.000337135 9.91161e-05 30
produce:reco3d:SpacePointSolver 0.244595 1.39383 8.76095 0.765539 1.66149 30
produce:hitpdune:DisambigFromSpacePoints 0.109987 0.646907 1.60988 0.59957 0.354243 30
produce:pandora:StandardPandora 6.74099 16.4294 33.2942 15.5919 6.30749 30
produce:pandoraWriter:StandardPandora 0.0678521 0.166746 0.246246 0.15925 0.0380045 30
produce:pandoraTrack:LArPandoraTrackCreation 0.272424 1.5817 3.0843 1.54064 0.563597 30
produce:pandoraShower:LArPandoraShowerCreation 0.228311 1.87946 3.95736 1.7479 0.815037 30
produce:pandoracalo:Calorimetry 0.185457 0.827637 1.58446 0.791046 0.280275 30
produce:pandoracalonosce:Calorimetry 0.153093 0.755147 1.55347 0.728084 0.268341 30
produce:pandorapid:Chi2ParticleID 0.00152057 0.00196158 0.00262335 0.00195268 0.000275751 30
produce:pandoraShowercalo:ShowerCalorimetry 0.208407 1.75952 3.84803 1.64417 0.772113 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.202187 1.71244 3.65545 1.60849 0.756196 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0710856 0.149034 0.241041 0.147589 0.031694 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0228063 0.0558083 0.0913908 0.0549269 0.0128864 30
produce:emtrkmichelid:EmTrackMichelId 15.4008 31.3403 51.5872 31.0336 7.11375 30
[art]:TriggerResults:TriggerResultInserter 2.4544e-05 3.39913e-05 0.000105439 3.02745e-05 1.46963e-05 30
end_path:out1:RootOutput 6.71e-06 8.68693e-06 3.108e-05 7.778e-06 4.20794e-06 30
end_path:michelremoving:michelremoving 0.0255129 0.0707353 0.242917 0.0456111 0.0569127 30
end_path:michelremovinggnocchi:michelremoving 0.0188824 0.0678652 0.240127 0.0395223 0.057633 30
end_path:calibanagnocchi:CalibAnaTree 0.124661 0.587545 1.08537 0.56947 0.194833 30
end_path:out1:RootOutput(write) 0.385573 0.850042 1.28193 0.827654 0.187479 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4959.47 MB
Peak resident set size usage (VmHWM): 2357.85 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0766_dataflow4_datawriter_0_20240907T032948_reco_stage1_reco_stage2_20240926T155914_keepup.root
\tHists: np04hd_raw_run029107_0766_dataflow4_datawriter_0_20240907T032948_reco_stage1_reco_stage2_20240926T155914_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
