Jobsub ID 78440.187@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78440.187@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:39:18 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 11:03:41 |
| From worker node | Hostname | ccwcondor0559 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:54:28 |
| Input files | hd-protodune:np04hd_raw_run029081_0429_dataflow5_datawriter_0_20240904T234612.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:03:41 |
| Saved logs | justin-logs:78440.187-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
063e+06
1 7.25234e+06
2 7.2396e+06
3 7.23543e+06
Now with regularization...
Begin: 6.91767e+06
0 6.91514e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 103181 TrkackID 0 startx, y, z 25.1588 413.908 453.954 wireno, peakt tpcno 0 1779.29 5 dist 202.644min T, max_T5.04043 4257.58
no of hits closeby 0 event 103181 TrkackID 0 startx, y, z 25.1588 413.908 453.954 wireno, peakt tpcno 0 1779.29 5 dist 202.644min T, max_T5.04043 4257.58
Begin processing the 29th record. run: 29081 subRun: 1 event: 103237 at 26-Sep-2024 12:51:04 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 81373 traces, 81373 summary
Retagger: tagging trace set: gauss with 60405 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 60405 traces tagged "gauss"
FrameSaver: q=1.28816e+06 n=1087243 tag=gauss
wclsFrameSaver: saving 81373 traces tagged "wiener"
FrameSaver: q=1.41119e+06 n=984662 tag=wiener
1937 hits in TPC 0
2902 hits in TPC 1
5961 hits in TPC 2
3768 hits in TPC 3
5520 hits in TPC 4
10471 hits in TPC 5
7745 hits in TPC 6
4337 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 81 XUs and 78 XVs -> 11 XUVs
C:0 T:2 41654 XUs and 72682 XVs -> 2272 XUVs
C:0 T:3 74 XUs and 132 XVs -> 3 XUVs
C:0 T:4 244 XUs and 479 XVs -> 7 XUVs
C:0 T:5 131031 XUs and 111910 XVs -> 4848 XUVs
C:0 T:6 92361 XUs and 78345 XVs -> 999 XUVs
C:0 T:7 216 XUs and 230 XVs -> 8 XUVs
8148 XUVs total
2712 collection wire objects
8148 potential space points
Neighbour search...
358780 tests to find 193262 neighbours
Iterating with no regularization...
Begin: 2.50881e+07
0 2.05989e+07
1 2.03222e+07
2 2.02778e+07
3 2.02624e+07
Now with regularization...
Begin: 1.96067e+07
0 1.95971e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 7 event 103237 TrkackID 13 startx, y, z 75.636 165.152 443.116 wireno, peakt tpcno 159 3388.24 5 dist 1.149min T, max_T2791.37 3853.42
no of hits closeby 7 event 103237 TrkackID 13 startx, y, z 75.636 165.152 443.116 wireno, peakt tpcno 159 3388.24 5 dist 1.149min T, max_T2791.37 3853.42
Begin processing the 30th record. run: 29081 subRun: 1 event: 103245 at 26-Sep-2024 12:52:32 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 71988 traces, 71988 summary
Retagger: tagging trace set: gauss with 54209 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54209 traces tagged "gauss"
FrameSaver: q=1.24346e+06 n=1042595 tag=gauss
wclsFrameSaver: saving 71988 traces tagged "wiener"
FrameSaver: q=1.33406e+06 n=948342 tag=wiener
2604 hits in TPC 0
2437 hits in TPC 1
5592 hits in TPC 2
4894 hits in TPC 3
5058 hits in TPC 4
10693 hits in TPC 5
5227 hits in TPC 6
2594 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 61 XUs and 58 XVs -> 3 XUVs
C:0 T:1 2725 XUs and 3681 XVs -> 1103 XUVs
C:0 T:2 13194 XUs and 12363 XVs -> 833 XUVs
C:0 T:3 144 XUs and 205 XVs -> 1 XUVs
C:0 T:4 222 XUs and 186 XVs -> 6 XUVs
C:0 T:5 47364 XUs and 50425 XVs -> 2023 XUVs
C:0 T:6 14623 XUs and 28828 XVs -> 837 XUVs
C:0 T:7 143 XUs and 214 XVs -> 3 XUVs
4809 XUVs total
2322 collection wire objects
4809 potential space points
Neighbour search...
189301 tests to find 94414 neighbours
Iterating with no regularization...
Begin: 1.461e+07
0 1.34246e+07
1 1.33807e+07
2 1.3377e+07
Now with regularization...
Begin: 1.26114e+07
0 1.26098e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:53:56 CEST Closed output file "np04hd_raw_run029081_0429_dataflow5_datawriter_0_20240904T234612_reco_stage1_reco_stage2_20240926T105356_keepup.root"
26-Sep-2024 12:53:56 CEST Closed input file "np04hd_raw_run029081_0429_dataflow5_datawriter_0_20240904T234612_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 52.9911 84.5131 137.971 83.5064 16.1829 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000620768 0.00248654 0.00462535 0.00272906 0.00117198 30
produce:fembfilter:PDHDFEMBFilter 0.497996 0.568646 0.747487 0.523244 0.0820253 30
produce:wclsdatahd:WireCellToolkit 29.9881 34.8471 41.6146 33.9105 3.565 30
produce:gaushit:GausHitFinder 0.336186 0.612223 0.907665 0.585112 0.146053 30
produce:nhitsfilter:NumberOfHitsFilter 0.000212718 0.000318617 0.00047683 0.000308146 6.58215e-05 30
produce:reco3d:SpacePointSolver 0.163085 0.928068 4.51349 0.615524 0.936302 30
produce:hitpdune:DisambigFromSpacePoints 0.120371 0.449037 1.089 0.446091 0.20497 30
produce:pandora:StandardPandora 4.57764 10.2019 24.7782 9.97884 3.90645 30
produce:pandoraWriter:StandardPandora 0.0652766 0.130556 0.199484 0.127481 0.0304299 30
produce:pandoraTrack:LArPandoraTrackCreation 0.433634 1.52501 3.53777 1.54345 0.629307 30
produce:pandoraShower:LArPandoraShowerCreation 0.397107 1.94504 5.37265 1.9548 0.991945 30
produce:pandoracalo:Calorimetry 0.239168 0.769179 1.58159 0.774296 0.286065 30
produce:pandoracalonosce:Calorimetry 0.228195 0.697164 1.38641 0.749117 0.240119 30
produce:pandorapid:Chi2ParticleID 0.00114515 0.00192137 0.00360043 0.00172878 0.000554317 30
produce:pandoraShowercalo:ShowerCalorimetry 0.373101 1.74233 4.68919 1.67886 0.853283 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.38957 1.609 4.4863 1.536 0.774831 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0587219 0.114049 0.238823 0.10555 0.0371051 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0226076 0.0445713 0.0774461 0.0416542 0.0129039 30
produce:emtrkmichelid:EmTrackMichelId 14.3481 26.8647 45.9119 26.4303 6.35243 30
[art]:TriggerResults:TriggerResultInserter 2.153e-05 5.50606e-05 0.000120876 5.32395e-05 1.93258e-05 30
end_path:out1:RootOutput 1.083e-05 1.70816e-05 3.0126e-05 1.71775e-05 4.46925e-06 30
end_path:michelremoving:michelremoving 0.0190762 0.0705317 0.234118 0.044583 0.0551691 30
end_path:michelremovinggnocchi:michelremoving 0.0209885 0.0643747 0.212401 0.0374905 0.0518253 30
end_path:calibanagnocchi:CalibAnaTree 0.178511 0.516353 1.03239 0.477055 0.184329 30
end_path:out1:RootOutput(write) 0.345489 0.619003 1.11856 0.607809 0.147271 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4964.07 MB
Peak resident set size usage (VmHWM): 2371.85 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0429_dataflow5_datawriter_0_20240904T234612_reco_stage1_reco_stage2_20240926T105356_keepup.root
\tHists: np04hd_raw_run029081_0429_dataflow5_datawriter_0_20240904T234612_reco_stage1_reco_stage2_20240926T105356_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
