Jobsub ID 263294.0@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263294.0@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3452 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:58:33 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 11:20:03 |
| From worker node | Hostname | ccwcondor0788 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:01:44 |
| Input files | hd-protodune:np04hd_raw_run029107_2661_dataflow0_datawriter_0_20240908T114117.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:20:03 |
| Saved logs | justin-logs:263294.0-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
XUs and 4903 XVs -> 361 XUVs
C:0 T:2 10555 XUs and 8070 XVs -> 290 XUVs
C:0 T:3 42 XUs and 76 XVs -> 1 XUVs
C:0 T:4 140 XUs and 239 XVs -> 5 XUVs
C:0 T:5 34505 XUs and 30038 XVs -> 816 XUVs
C:0 T:6 15926 XUs and 10102 XVs -> 325 XUVs
C:0 T:7 163 XUs and 131 XVs -> 2 XUVs
1800 XUVs total
951 collection wire objects
1800 potential space points
Neighbour search...
55588 tests to find 26466 neighbours
Iterating with no regularization...
Begin: 4.84912e+06
0 4.66585e+06
1 4.65955e+06
2 4.65891e+06
Now with regularization...
Begin: 4.36499e+06
0 4.36474e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 14 event 638888 TrkackID 1 startx, y, z 183.575 537.569 203.867 wireno, peakt tpcno 394 2288 2 dist 341.958min T, max_T5.1019 2326.61
no of hits closeby 14 event 638888 TrkackID 1 startx, y, z 183.575 537.569 203.867 wireno, peakt tpcno 394 2288 2 dist 341.958min T, max_T5.1019 2326.61
Begin processing the 29th record. run: 29107 subRun: 1 event: 638896 at 26-Sep-2024 13:04:24 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 51815 traces, 51815 summary
Retagger: tagging trace set: gauss with 37220 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 37220 traces tagged "gauss"
FrameSaver: q=823920 n=675268 tag=gauss
wclsFrameSaver: saving 51815 traces tagged "wiener"
FrameSaver: q=890912 n=613017 tag=wiener
2428 hits in TPC 0
2246 hits in TPC 1
5430 hits in TPC 2
3868 hits in TPC 3
1626 hits in TPC 4
2753 hits in TPC 5
5536 hits in TPC 6
2656 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 25 XUs and 26 XVs -> 0 XUVs
C:0 T:2 19646 XUs and 17363 XVs -> 2312 XUVs
C:0 T:3 66 XUs and 50 XVs -> 2 XUVs
C:0 T:4 160 XUs and 276 XVs -> 11 XUVs
C:0 T:5 3057 XUs and 4651 XVs -> 207 XUVs
C:0 T:6 17229 XUs and 24270 XVs -> 671 XUVs
C:0 T:7 290 XUs and 403 XVs -> 20 XUVs
3223 XUVs total
1204 collection wire objects
3223 potential space points
Neighbour search...
73251 tests to find 46646 neighbours
Iterating with no regularization...
Begin: 5.92128e+06
0 5.39766e+06
1 5.35488e+06
2 5.34849e+06
3 5.34693e+06
Now with regularization...
Begin: 4.71815e+06
0 4.71722e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 638920 at 26-Sep-2024 13:05:35 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 94016 traces, 94016 summary
Retagger: tagging trace set: gauss with 66397 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 66397 traces tagged "gauss"
FrameSaver: q=1.38936e+06 n=1140182 tag=gauss
wclsFrameSaver: saving 94016 traces tagged "wiener"
FrameSaver: q=1.52217e+06 n=1023104 tag=wiener
4084 hits in TPC 0
4439 hits in TPC 1
8265 hits in TPC 2
4110 hits in TPC 3
4279 hits in TPC 4
8796 hits in TPC 5
8511 hits in TPC 6
3828 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 416 XUs and 365 XVs -> 5 XUVs
C:0 T:2 37989 XUs and 54552 XVs -> 1384 XUVs
C:0 T:3 222 XUs and 392 XVs -> 42 XUVs
C:0 T:4 191 XUs and 181 XVs -> 4 XUVs
C:0 T:5 44074 XUs and 45364 XVs -> 1399 XUVs
C:0 T:6 58935 XUs and 65204 XVs -> 1735 XUVs
C:0 T:7 796 XUs and 805 XVs -> 16 XUVs
4585 XUVs total
2178 collection wire objects
4585 potential space points
Neighbour search...
187145 tests to find 103578 neighbours
Iterating with no regularization...
Begin: 1.52787e+07
0 1.41299e+07
1 1.40302e+07
2 1.40232e+07
Now with regularization...
Begin: 1.33481e+07
0 1.33449e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:07:08 CEST Closed output file "np04hd_raw_run029107_2661_dataflow0_datawriter_0_20240908T114117_reco_stage1_reco_stage2_20240926T110708_keepup.root"
26-Sep-2024 13:07:08 CEST Closed input file "np04hd_raw_run029107_2661_dataflow0_datawriter_0_20240908T114117_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 49.1667 93.1686 162.257 87.563 25.0371 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000582318 0.0026519 0.00522417 0.00272694 0.00124073 30
produce:fembfilter:PDHDFEMBFilter 0.491705 0.641114 0.777483 0.657791 0.107474 30
produce:wclsdatahd:WireCellToolkit 31.9492 39.0762 46.3569 39.1303 4.79527 30
produce:gaushit:GausHitFinder 0.237636 0.644899 1.27235 0.569646 0.248297 30
produce:nhitsfilter:NumberOfHitsFilter 0.000163505 0.000329262 0.000687214 0.000309017 0.000100803 30
produce:reco3d:SpacePointSolver 0.0739476 1.2731 6.21103 0.675951 1.33644 30
produce:hitpdune:DisambigFromSpacePoints 0.0632859 0.502852 1.09976 0.47155 0.248047 30
produce:pandora:StandardPandora 2.20137 11.8251 31.1118 9.53621 6.32846 30
produce:pandoraWriter:StandardPandora 0.0509858 0.147851 0.309304 0.140003 0.0552459 30
produce:pandoraTrack:LArPandoraTrackCreation 0.266295 1.55368 5.30334 1.18441 1.05097 30
produce:pandoraShower:LArPandoraShowerCreation 0.280361 1.82714 7.37749 1.3072 1.42507 30
produce:pandoracalo:Calorimetry 0.164202 0.741769 2.51329 0.620087 0.458182 30
produce:pandoracalonosce:Calorimetry 0.138508 0.694109 2.36452 0.53592 0.446443 30
produce:pandorapid:Chi2ParticleID 0.00082367 0.00188455 0.00489256 0.00173369 0.000781868 30
produce:pandoraShowercalo:ShowerCalorimetry 0.222058 1.66623 7.77366 1.18357 1.42926 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.214688 1.59605 7.20587 1.12397 1.29773 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0407355 0.131266 0.272515 0.123176 0.0523239 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0157798 0.0466083 0.104 0.0451836 0.0183294 30
produce:emtrkmichelid:EmTrackMichelId 11.8702 29.2398 47.7544 27.4517 8.83437 30
[art]:TriggerResults:TriggerResultInserter 2.5758e-05 4.6217e-05 0.000114364 4.1152e-05 1.76221e-05 30
end_path:out1:RootOutput 9.749e-06 1.55801e-05 5.9901e-05 1.2428e-05 9.17978e-06 30
end_path:michelremoving:michelremoving 0.0126944 0.0819779 0.437559 0.0398108 0.0965227 30
end_path:michelremovinggnocchi:michelremoving 0.0126235 0.0774538 0.425305 0.0352109 0.093935 30
end_path:calibanagnocchi:CalibAnaTree 0.100198 0.516397 1.44825 0.460619 0.29109 30
end_path:out1:RootOutput(write) 0.260782 0.694965 1.27865 0.655638 0.236486 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4927.98 MB
Peak resident set size usage (VmHWM): 2352.78 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2661_dataflow0_datawriter_0_20240908T114117_reco_stage1_reco_stage2_20240926T110708_keepup.root
\tHists: np04hd_raw_run029107_2661_dataflow0_datawriter_0_20240908T114117_reco_stage1_reco_stage2_20240926T110708_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
