Jobsub ID 78859.17@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78859.17@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:14:36 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:35:01 |
| From worker node | Hostname | n233.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7352 24-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:15:18 |
| Input files | hd-protodune:np04hd_raw_run029107_2250_dataflow0_datawriter_0_20240908T042045.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:35:01 |
| Saved logs | justin-logs:78859.17-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
: 1.11546e+07
0 1.1152e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 540256 at 26-Sep-2024 15:30:39 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 77783 traces, 77783 summary
Retagger: tagging trace set: gauss with 56257 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56257 traces tagged "gauss"
FrameSaver: q=1.29177e+06 n=1056590 tag=gauss
wclsFrameSaver: saving 77783 traces tagged "wiener"
FrameSaver: q=1.39574e+06 n=957760 tag=wiener
3300 hits in TPC 0
2606 hits in TPC 1
5736 hits in TPC 2
4040 hits in TPC 3
5893 hits in TPC 4
7993 hits in TPC 5
6552 hits in TPC 6
5243 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 518 XUs and 275 XVs -> 10 XUVs
C:0 T:2 25753 XUs and 20822 XVs -> 3646 XUVs
C:0 T:3 2205 XUs and 3844 XVs -> 215 XUVs
C:0 T:4 503 XUs and 354 XVs -> 17 XUVs
C:0 T:5 52185 XUs and 66755 XVs -> 7736 XUVs
C:0 T:6 31516 XUs and 48778 XVs -> 2107 XUVs
C:0 T:7 4689 XUs and 6653 XVs -> 287 XUVs
14018 XUVs total
1929 collection wire objects
14018 potential space points
Neighbour search...
2320168 tests to find 1172980 neighbours
Iterating with no regularization...
Begin: 1.37171e+07
0 1.15461e+07
1 1.13257e+07
2 1.12976e+07
3 1.12872e+07
Now with regularization...
Begin: 1.05589e+07
0 1.05463e+07
1 1.05379e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 540264 at 26-Sep-2024 15:32:00 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 42132 traces, 42132 summary
Retagger: tagging trace set: gauss with 32166 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 32166 traces tagged "gauss"
FrameSaver: q=679492 n=605819 tag=gauss
wclsFrameSaver: saving 42132 traces tagged "wiener"
FrameSaver: q=728904 n=554936 tag=wiener
2308 hits in TPC 0
1757 hits in TPC 1
3445 hits in TPC 2
1961 hits in TPC 3
2080 hits in TPC 4
4092 hits in TPC 5
3629 hits in TPC 6
2487 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 224 XUs and 43 XVs -> 1 XUVs
C:0 T:2 9262 XUs and 13747 XVs -> 219 XUVs
C:0 T:3 20 XUs and 43 XVs -> 2 XUVs
C:0 T:4 165 XUs and 240 XVs -> 17 XUVs
C:0 T:5 24690 XUs and 9303 XVs -> 399 XUVs
C:0 T:6 4885 XUs and 6952 XVs -> 169 XUVs
C:0 T:7 11 XUs and 11 XVs -> 0 XUVs
807 XUVs total
425 collection wire objects
807 potential space points
Neighbour search...
22043 tests to find 14534 neighbours
Iterating with no regularization...
Begin: 4.38366e+06
0 4.30498e+06
1 4.29843e+06
2 4.29764e+06
Now with regularization...
Begin: 4.1135e+06
0 4.11323e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 540264 TrkackID 1 startx, y, z 187.935 120.628 461.967 wireno, peakt tpcno 169 2706.77 5 dist 0min T, max_T2292.37 4637
no of hits closeby 1 event 540264 TrkackID 4 startx, y, z 34.7445 588.7 298.798 wireno, peakt tpcno 63 15 5 dist 0.922209min T, max_T5 1941.46
no of hits closeby 0 event 540264 TrkackID 1 startx, y, z 187.935 120.628 461.967 wireno, peakt tpcno 169 2706.77 5 dist 0min T, max_T2292.37 4637
no of hits closeby 1 event 540264 TrkackID 4 startx, y, z 34.7445 588.7 298.798 wireno, peakt tpcno 63 15 5 dist 0.922209min T, max_T5 1941.46
26-Sep-2024 15:32:53 BST Closed output file "np04hd_raw_run029107_2250_dataflow0_datawriter_0_20240908T042045_reco_stage1_reco_stage2_20240926T143253_keepup.root"
26-Sep-2024 15:32:53 BST Closed input file "np04hd_raw_run029107_2250_dataflow0_datawriter_0_20240908T042045_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 33.4008 69.4426 117.926 65.7538 16.3543 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000404908 0.00187961 0.00399702 0.00165007 0.000849779 30
produce:fembfilter:PDHDFEMBFilter 0.520087 0.590606 0.798021 0.582329 0.075557 30
produce:wclsdatahd:WireCellToolkit 28.0234 31.0205 39.2259 29.6095 3.04896 30
produce:gaushit:GausHitFinder 0.318229 0.590052 1.49092 0.551714 0.247026 30
produce:nhitsfilter:NumberOfHitsFilter 0.000239649 0.000382545 0.00102413 0.000364352 0.000140168 30
produce:reco3d:SpacePointSolver 0.196616 0.704594 1.89494 0.622599 0.402358 29
produce:hitpdune:DisambigFromSpacePoints 0.117702 0.397145 0.912969 0.375995 0.19675 29
produce:pandora:StandardPandora 4.16688 10.2491 29.8934 8.90568 5.50625 29
produce:pandoraWriter:StandardPandora 0.0728673 0.130103 0.244413 0.121442 0.0401799 29
produce:pandoraTrack:LArPandoraTrackCreation 0.29427 0.837374 1.8093 0.766697 0.367591 29
produce:pandoraShower:LArPandoraShowerCreation 0.353546 0.976117 2.43924 0.867546 0.501635 29
produce:pandoracalo:Calorimetry 0.180907 0.441558 1.06491 0.369286 0.192738 29
produce:pandoracalonosce:Calorimetry 0.151442 0.391277 0.991572 0.352747 0.178017 29
produce:pandorapid:Chi2ParticleID 0.000852766 0.00152273 0.00259126 0.00138453 0.000436479 29
produce:pandoraShowercalo:ShowerCalorimetry 0.291204 0.890692 3.04899 0.762007 0.551081 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.280972 0.86324 2.78055 0.734579 0.517892 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0766108 0.124008 0.227908 0.116122 0.0384606 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0259621 0.0411523 0.0706745 0.0387123 0.0121414 29
produce:emtrkmichelid:EmTrackMichelId 12.9729 21.1717 36.0705 19.7324 5.59558 29
[art]:TriggerResults:TriggerResultInserter 1.8374e-05 3.01946e-05 6.992e-05 2.6585e-05 1.2529e-05 30
end_path:out1:RootOutput 4.729e-06 9.64743e-06 2.0348e-05 8.8765e-06 2.90388e-06 30
end_path:michelremoving:michelremoving 2.0208e-05 0.061281 0.342167 0.0294339 0.0694825 30
end_path:michelremovinggnocchi:michelremoving 4.228e-06 0.0572363 0.304875 0.0265433 0.0656124 30
end_path:calibanagnocchi:CalibAnaTree 4.388e-06 0.3314 0.715935 0.277732 0.15813 30
end_path:out1:RootOutput(write) 0.44289 0.725916 1.23497 0.662037 0.194955 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4956.74 MB
Peak resident set size usage (VmHWM): 2624.96 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2250_dataflow0_datawriter_0_20240908T042045_reco_stage1_reco_stage2_20240926T143253_keepup.root
\tHists: np04hd_raw_run029107_2250_dataflow0_datawriter_0_20240908T042045_reco_stage1_reco_stage2_20240926T143253_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
