Jobsub ID 263276.80@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263276.80@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:54:33 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:14:57 |
| From worker node | Hostname | ccwcondor0625 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:55:29 |
| Input files | hd-protodune:np04hd_raw_run029092_0473_dataflow3_datawriter_0_20240906T004920.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:14:57 |
| Saved logs | justin-logs:263276.80-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
.39742e+06
0 4.39635e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29092 subRun: 1 event: 113755 at 26-Sep-2024 13:01:09 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 79936 traces, 79936 summary
Retagger: tagging trace set: gauss with 57024 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 57024 traces tagged "gauss"
FrameSaver: q=1.21791e+06 n=1004112 tag=gauss
wclsFrameSaver: saving 79936 traces tagged "wiener"
FrameSaver: q=1.31616e+06 n=892835 tag=wiener
4192 hits in TPC 0
3972 hits in TPC 1
7490 hits in TPC 2
3442 hits in TPC 3
2991 hits in TPC 4
4921 hits in TPC 5
8274 hits in TPC 6
4502 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 756 XUs and 776 XVs -> 15 XUVs
C:0 T:2 45693 XUs and 53587 XVs -> 2062 XUVs
C:0 T:3 166 XUs and 261 XVs -> 10 XUVs
C:0 T:4 3 XUs and 3 XVs -> 0 XUVs
C:0 T:5 18107 XUs and 19616 XVs -> 801 XUVs
C:0 T:6 32206 XUs and 55614 XVs -> 3900 XUVs
C:0 T:7 31 XUs and 44 XVs -> 0 XUVs
6788 XUVs total
2071 collection wire objects
6788 potential space points
Neighbour search...
680518 tests to find 315086 neighbours
Iterating with no regularization...
Begin: 8.45403e+06
0 7.72723e+06
1 7.68247e+06
2 7.67634e+06
Now with regularization...
Begin: 6.8146e+06
0 6.81232e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 113755 TrkackID 4 startx, y, z 80.1498 595.74 187.276 wireno, peakt tpcno 299 8.64186 5 dist 0min T, max_T1.51775 1790.3
no of hits closeby 1 event 113755 TrkackID 4 startx, y, z 80.1498 595.74 187.276 wireno, peakt tpcno 299 8.64186 5 dist 0min T, max_T1.51775 1790.3
Begin processing the 30th record. run: 29092 subRun: 1 event: 113763 at 26-Sep-2024 13:02:46 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 44517 traces, 44517 summary
Retagger: tagging trace set: gauss with 33412 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 33412 traces tagged "gauss"
FrameSaver: q=723392 n=634886 tag=gauss
wclsFrameSaver: saving 44517 traces tagged "wiener"
FrameSaver: q=764789 n=569733 tag=wiener
1900 hits in TPC 0
1730 hits in TPC 1
5632 hits in TPC 2
3196 hits in TPC 3
2704 hits in TPC 4
3925 hits in TPC 5
2761 hits in TPC 6
1796 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 208 XUs and 170 XVs -> 2 XUVs
C:0 T:2 15183 XUs and 11732 XVs -> 354 XUVs
C:0 T:3 227 XUs and 209 XVs -> 3 XUVs
C:0 T:4 144 XUs and 160 XVs -> 10 XUVs
C:0 T:5 7127 XUs and 6991 XVs -> 163 XUVs
C:0 T:6 3054 XUs and 5388 XVs -> 808 XUVs
C:0 T:7 309 XUs and 632 XVs -> 2 XUVs
1342 XUVs total
811 collection wire objects
1342 potential space points
Neighbour search...
15876 tests to find 8850 neighbours
Iterating with no regularization...
Begin: 3.29354e+06
0 3.17562e+06
1 3.17021e+06
2 3.17006e+06
Now with regularization...
Begin: 2.85302e+06
0 2.85294e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 4 event 113763 TrkackID 0 startx, y, z 121.091 364.287 6.14819 wireno, peakt tpcno 469 1243.69 5 dist 0.806391min T, max_T1236.74 3087.3
no of hits closeby 4 event 113763 TrkackID 0 startx, y, z 121.091 364.287 6.14819 wireno, peakt tpcno 469 1243.69 5 dist 0.806391min T, max_T1236.74 3087.3
26-Sep-2024 13:03:44 CEST Closed output file "np04hd_raw_run029092_0473_dataflow3_datawriter_0_20240906T004920_reco_stage1_reco_stage2_20240926T110344_keepup.root"
26-Sep-2024 13:03:44 CEST Closed input file "np04hd_raw_run029092_0473_dataflow3_datawriter_0_20240906T004920_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 53.9833 92.9917 128.974 95.6127 20.0309 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000559347 0.00308839 0.00446329 0.00311639 0.00104971 30
produce:fembfilter:PDHDFEMBFilter 0.495411 0.683928 0.832202 0.698995 0.0883142 30
produce:wclsdatahd:WireCellToolkit 30.4179 39.3136 44.5485 40.3839 3.75221 30
produce:gaushit:GausHitFinder 0.332706 0.684881 1.02381 0.723702 0.207973 30
produce:nhitsfilter:NumberOfHitsFilter 0.000224861 0.000347973 0.000584705 0.000346449 7.61829e-05 30
produce:reco3d:SpacePointSolver 0.0901497 1.24392 7.41789 0.839898 1.4307 30
produce:hitpdune:DisambigFromSpacePoints 0.141108 0.510798 1.21961 0.530326 0.276864 30
produce:pandora:StandardPandora 4.21386 13.035 38.5845 11.9978 6.9635 30
produce:pandoraWriter:StandardPandora 0.0719281 0.162979 0.301553 0.15887 0.0448609 30
produce:pandoraTrack:LArPandoraTrackCreation 0.622135 1.91832 3.52986 1.96095 0.816309 30
produce:pandoraShower:LArPandoraShowerCreation 0.354711 2.36722 6.28215 2.05281 1.43635 30
produce:pandoracalo:Calorimetry 0.29751 0.892439 1.93165 0.881951 0.426616 30
produce:pandoracalonosce:Calorimetry 0.235231 0.83017 1.71703 0.852161 0.393067 30
produce:pandorapid:Chi2ParticleID 0.00114327 0.00224173 0.0035567 0.00244079 0.000722631 30
produce:pandoraShowercalo:ShowerCalorimetry 0.302289 1.96285 5.01084 1.78238 1.17694 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.315082 1.8779 5.13688 1.67486 1.16469 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0552421 0.149199 0.265264 0.139008 0.0506372 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.021711 0.0518251 0.093859 0.0511286 0.0186386 30
produce:emtrkmichelid:EmTrackMichelId 12.7819 25.6515 38.2128 25.9269 6.29878 30
[art]:TriggerResults:TriggerResultInserter 5.0294e-05 7.01727e-05 9.5428e-05 7.05025e-05 1.09839e-05 30
end_path:out1:RootOutput 1.2163e-05 2.06396e-05 3.3824e-05 2.0954e-05 3.9662e-06 30
end_path:michelremoving:michelremoving 0.018822 0.0799464 0.354431 0.0474407 0.0774446 30
end_path:michelremovinggnocchi:michelremoving 0.0160015 0.0767753 0.368106 0.0409053 0.0793644 30
end_path:calibanagnocchi:CalibAnaTree 0.166689 0.559765 0.936051 0.567224 0.210384 30
end_path:out1:RootOutput(write) 0.337653 0.707432 1.12042 0.67852 0.185953 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4918.22 MB
Peak resident set size usage (VmHWM): 2349.47 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0473_dataflow3_datawriter_0_20240906T004920_reco_stage1_reco_stage2_20240926T110344_keepup.root
\tHists: np04hd_raw_run029092_0473_dataflow3_datawriter_0_20240906T004920_reco_stage1_reco_stage2_20240926T110344_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
