Jobsub ID 263340.3@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263340.3@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 10:13:50 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 12:23:46 |
| From worker node | Hostname | cn-leo-118.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:45:44 |
| Input files | hd-protodune:np04hd_raw_run029107_0833_dataflow2_datawriter_0_20240907T042312.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 12:23:46 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 5 event 200162 TrkackID 6 startx, y, z 80.165 407.906 408.88 wireno, peakt tpcno 172 48.4664 5 dist 6.08033min T, max_T4.81999 1027.88
no of hits closeby 5 event 200162 TrkackID 6 startx, y, z 80.165 407.906 408.88 wireno, peakt tpcno 172 48.4664 5 dist 6.08033min T, max_T4.81999 1027.88
Begin processing the 29th record. run: 29107 subRun: 1 event: 200170 at 26-Sep-2024 13:52:53 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 71359 traces, 71359 summary
Retagger: tagging trace set: gauss with 54752 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54752 traces tagged "gauss"
FrameSaver: q=1.17979e+06 n=1058131 tag=gauss
wclsFrameSaver: saving 71359 traces tagged "wiener"
FrameSaver: q=1.27634e+06 n=966569 tag=wiener
5481 hits in TPC 0
5425 hits in TPC 1
4317 hits in TPC 2
3050 hits in TPC 3
4283 hits in TPC 4
8336 hits in TPC 5
5908 hits in TPC 6
2612 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 9527 XUs and 20562 XVs -> 601 XUVs
C:0 T:2 8534 XUs and 7413 XVs -> 471 XUVs
C:0 T:3 1246 XUs and 2387 XVs -> 56 XUVs
C:0 T:4 124 XUs and 153 XVs -> 7 XUVs
C:0 T:5 41632 XUs and 58979 XVs -> 1659 XUVs
C:0 T:6 18058 XUs and 20700 XVs -> 774 XUVs
C:0 T:7 715 XUs and 465 XVs -> 11 XUVs
3579 XUVs total
1456 collection wire objects
3579 potential space points
Neighbour search...
134743 tests to find 77878 neighbours
Iterating with no regularization...
Begin: 9.55353e+06
0 9.15174e+06
1 9.12608e+06
2 9.12222e+06
Now with regularization...
Begin: 8.52188e+06
0 8.52029e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 200218 at 26-Sep-2024 13:54:53 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58001 traces, 58001 summary
Retagger: tagging trace set: gauss with 43489 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43489 traces tagged "gauss"
FrameSaver: q=1.22154e+06 n=883077 tag=gauss
wclsFrameSaver: saving 58001 traces tagged "wiener"
FrameSaver: q=1.36644e+06 n=818421 tag=wiener
10276 hits in TPC 0
6972 hits in TPC 1
3241 hits in TPC 2
2169 hits in TPC 3
2606 hits in TPC 4
5496 hits in TPC 5
3138 hits in TPC 6
1976 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 37060 XUs and 41509 XVs -> 5234 XUVs
C:0 T:2 9613 XUs and 6066 XVs -> 212 XUVs
C:0 T:3 14 XUs and 10 XVs -> 0 XUVs
C:0 T:4 27 XUs and 42 XVs -> 0 XUVs
C:0 T:5 14790 XUs and 47565 XVs -> 980 XUVs
C:0 T:6 4867 XUs and 4852 XVs -> 264 XUVs
C:0 T:7 299 XUs and 421 XVs -> 27 XUVs
6717 XUVs total
1249 collection wire objects
6717 potential space points
Neighbour search...
662031 tests to find 236448 neighbours
Iterating with no regularization...
Begin: 1.45863e+08
0 1.37152e+08
1 1.35866e+08
2 1.35855e+08
Now with regularization...
Begin: 1.3555e+08
0 1.35548e+08
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:57:04 CEST Closed output file "np04hd_raw_run029107_0833_dataflow2_datawriter_0_20240907T042312_reco_stage1_reco_stage2_20240926T115704_keepup.root"
26-Sep-2024 13:57:04 CEST Closed input file "np04hd_raw_run029107_0833_dataflow2_datawriter_0_20240907T042312_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 75.4916 103.299 143.375 101.032 15.8506 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00061144 0.00289079 0.00754599 0.00203568 0.0022236 30
produce:fembfilter:PDHDFEMBFilter 0.700807 0.728197 0.871483 0.720624 0.0351019 30
produce:wclsdatahd:WireCellToolkit 54.9776 60.3584 69.0806 60.1211 2.88769 30
produce:gaushit:GausHitFinder 0.356336 0.594506 0.858026 0.604016 0.129497 30
produce:nhitsfilter:NumberOfHitsFilter 0.000374062 0.000516762 0.000701183 0.000523827 7.2587e-05 30
produce:reco3d:SpacePointSolver 0.287679 1.47106 5.12832 1.38805 1.05947 30
produce:hitpdune:DisambigFromSpacePoints 0.140173 0.367104 0.640891 0.356971 0.129089 30
produce:pandora:StandardPandora 4.32563 11.2158 29.3783 9.75772 5.14867 30
produce:pandoraWriter:StandardPandora 0.0936549 0.171399 0.271814 0.16642 0.0398853 30
produce:pandoraTrack:LArPandoraTrackCreation 0.560022 2.69265 6.12115 2.5079 1.24947 30
produce:pandoraShower:LArPandoraShowerCreation 0.629084 3.6605 11.4959 2.93593 2.22716 30
produce:pandoracalo:Calorimetry 0.283417 1.0298 2.20453 0.924228 0.4126 30
produce:pandoracalonosce:Calorimetry 0.256998 0.967108 2.29908 0.872362 0.421506 30
produce:pandorapid:Chi2ParticleID 0.00113762 0.00167781 0.0021738 0.00163381 0.00028054 30
produce:pandoraShowercalo:ShowerCalorimetry 0.53808 2.55294 7.78863 2.18059 1.34173 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.501866 2.45036 7.54756 2.13243 1.28103 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0594865 0.102015 0.153462 0.0977163 0.0214338 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0229612 0.0461807 0.0742202 0.0447932 0.010235 30
produce:emtrkmichelid:EmTrackMichelId 8.16591 12.8906 19.8887 12.7897 2.45911 30
[art]:TriggerResults:TriggerResultInserter 4.4453e-05 7.17212e-05 9.8625e-05 7.4265e-05 1.492e-05 30
end_path:out1:RootOutput 1.7318e-05 2.71953e-05 5.8575e-05 2.65615e-05 8.24953e-06 30
end_path:michelremoving:michelremoving 0.0238166 0.103405 0.292984 0.0604118 0.074976 30
end_path:michelremovinggnocchi:michelremoving 0.0172043 0.100144 0.283931 0.0593074 0.0737144 30
end_path:calibanagnocchi:CalibAnaTree 0.176257 0.728266 1.55454 0.682072 0.300608 30
end_path:out1:RootOutput(write) 0.402084 0.651552 0.931574 0.630454 0.13776 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5823.11 MB
Peak resident set size usage (VmHWM): 2660.57 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0833_dataflow2_datawriter_0_20240907T042312_reco_stage1_reco_stage2_20240926T115704_keepup.root
\tHists: np04hd_raw_run029107_0833_dataflow2_datawriter_0_20240907T042312_reco_stage1_reco_stage2_20240926T115704_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
