Jobsub ID 78452.31@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78452.31@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:55:43 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 11:08:07 |
| From worker node | Hostname | ccwcondor0586 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:00:50 |
| Input files | hd-protodune:np04hd_raw_run029092_0315_dataflow7_datawriter_0_20240905T214906.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:08:07 |
| Saved logs | justin-logs:78452.31-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
XUs and 37358 XVs -> 1211 XUVs
C:0 T:3 203 XUs and 272 XVs -> 40 XUVs
C:0 T:4 40 XUs and 54 XVs -> 0 XUVs
C:0 T:5 14794 XUs and 21023 XVs -> 785 XUVs
C:0 T:6 64408 XUs and 57126 XVs -> 1945 XUVs
C:0 T:7 634 XUs and 561 XVs -> 9 XUVs
4007 XUVs total
2150 collection wire objects
4007 potential space points
Neighbour search...
86591 tests to find 52212 neighbours
Iterating with no regularization...
Begin: 1.5834e+07
0 1.44011e+07
1 1.43189e+07
2 1.43113e+07
Now with regularization...
Begin: 1.34357e+07
0 1.3433e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 75823 TrkackID 2 startx, y, z 320.681 395.776 461.752 wireno, peakt tpcno 128 3053.54 5 dist 0min T, max_T136.92 4196.5
no of hits closeby 0 event 75823 TrkackID 2 startx, y, z 320.681 395.776 461.752 wireno, peakt tpcno 128 3053.54 5 dist 0min T, max_T136.92 4196.5
Begin processing the 29th record. run: 29092 subRun: 1 event: 75831 at 26-Sep-2024 12:56:07 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 50408 traces, 50408 summary
Retagger: tagging trace set: gauss with 37750 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 37750 traces tagged "gauss"
FrameSaver: q=738327 n=675928 tag=gauss
wclsFrameSaver: saving 50408 traces tagged "wiener"
FrameSaver: q=799497 n=613537 tag=wiener
2917 hits in TPC 0
2357 hits in TPC 1
3285 hits in TPC 2
1364 hits in TPC 3
4997 hits in TPC 4
6966 hits in TPC 5
3115 hits in TPC 6
1790 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 198 XUs and 128 XVs -> 1 XUVs
C:0 T:2 6985 XUs and 9373 XVs -> 266 XUVs
C:0 T:3 29 XUs and 35 XVs -> 1 XUVs
C:0 T:4 1717 XUs and 3261 XVs -> 201 XUVs
C:0 T:5 38524 XUs and 35699 XVs -> 921 XUVs
C:0 T:6 5454 XUs and 8890 XVs -> 246 XUVs
C:0 T:7 302 XUs and 403 XVs -> 23 XUVs
1659 XUVs total
886 collection wire objects
1659 potential space points
Neighbour search...
51549 tests to find 28940 neighbours
Iterating with no regularization...
Begin: 4.49393e+06
0 4.26041e+06
1 4.23998e+06
2 4.23833e+06
Now with regularization...
Begin: 4.03729e+06
0 4.03663e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29092 subRun: 1 event: 75839 at 26-Sep-2024 12:57:15 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 74822 traces, 74822 summary
Retagger: tagging trace set: gauss with 56615 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56615 traces tagged "gauss"
FrameSaver: q=1.30459e+06 n=1091252 tag=gauss
wclsFrameSaver: saving 74822 traces tagged "wiener"
FrameSaver: q=1.38495e+06 n=988948 tag=wiener
2996 hits in TPC 0
2073 hits in TPC 1
7511 hits in TPC 2
5431 hits in TPC 3
5155 hits in TPC 4
5316 hits in TPC 5
7812 hits in TPC 6
4347 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 392 XUs and 219 XVs -> 5 XUVs
C:0 T:2 28543 XUs and 28520 XVs -> 697 XUVs
C:0 T:3 317 XUs and 346 XVs -> 9 XUVs
C:0 T:4 63 XUs and 113 XVs -> 4 XUVs
C:0 T:5 25090 XUs and 17522 XVs -> 957 XUVs
C:0 T:6 20048 XUs and 33843 XVs -> 1605 XUVs
C:0 T:7 43 XUs and 54 XVs -> 2 XUVs
3279 XUVs total
1776 collection wire objects
3279 potential space points
Neighbour search...
67199 tests to find 35182 neighbours
Iterating with no regularization...
Begin: 8.4739e+06
0 7.46795e+06
1 7.40572e+06
2 7.4004e+06
Now with regularization...
Begin: 6.7039e+06
0 6.70238e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:58:35 CEST Closed output file "np04hd_raw_run029092_0315_dataflow7_datawriter_0_20240905T214906_reco_stage1_reco_stage2_20240926T105835_keepup.root"
26-Sep-2024 12:58:35 CEST Closed input file "np04hd_raw_run029092_0315_dataflow7_datawriter_0_20240905T214906_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 60.2671 81.3148 108.342 81.2955 13.1199 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000421406 0.00261951 0.00484527 0.00282559 0.00121702 30
produce:fembfilter:PDHDFEMBFilter 0.489341 0.573175 0.776807 0.535453 0.0850812 30
produce:wclsdatahd:WireCellToolkit 29.9449 34.636 41.826 35.042 2.94292 30
produce:gaushit:GausHitFinder 0.317052 0.573804 0.91206 0.583383 0.126794 30
produce:nhitsfilter:NumberOfHitsFilter 0.000213218 0.000313453 0.000628404 0.000302134 7.78222e-05 30
produce:reco3d:SpacePointSolver 0.0693337 0.850089 2.25927 0.794891 0.493751 30
produce:hitpdune:DisambigFromSpacePoints 0.0814923 0.385323 0.741392 0.353308 0.168822 30
produce:pandora:StandardPandora 3.47977 10.0002 16.921 10.0485 3.47708 30
produce:pandoraWriter:StandardPandora 0.0732356 0.135457 0.211645 0.13496 0.0380152 30
produce:pandoraTrack:LArPandoraTrackCreation 0.653782 1.39413 2.86759 1.33162 0.565869 30
produce:pandoraShower:LArPandoraShowerCreation 0.536354 1.61744 4.31948 1.49815 0.907989 30
produce:pandoracalo:Calorimetry 0.281708 0.666581 1.42333 0.618013 0.257816 30
produce:pandoracalonosce:Calorimetry 0.24558 0.61024 1.57364 0.551377 0.271978 30
produce:pandorapid:Chi2ParticleID 0.00114296 0.0017474 0.00352814 0.00161911 0.000549942 30
produce:pandoraShowercalo:ShowerCalorimetry 0.356402 1.35066 3.72058 1.25342 0.79684 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.402741 1.32496 3.57163 1.231 0.791262 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0556837 0.107329 0.188907 0.105016 0.0316407 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0198784 0.0388637 0.0657661 0.0372899 0.0108489 30
produce:emtrkmichelid:EmTrackMichelId 13.8177 25.5825 36.6419 26.6365 5.63485 30
[art]:TriggerResults:TriggerResultInserter 2.9765e-05 6.11424e-05 0.000147015 5.81885e-05 2.31997e-05 30
end_path:out1:RootOutput 8.977e-06 1.74439e-05 3.9585e-05 1.618e-05 6.36917e-06 30
end_path:michelremoving:michelremoving 0.0162284 0.0931632 0.293316 0.0532975 0.0756262 30
end_path:michelremovinggnocchi:michelremoving 0.0152429 0.0900882 0.373605 0.0472176 0.0798741 30
end_path:calibanagnocchi:CalibAnaTree 0.194212 0.481534 1.23704 0.430341 0.227565 30
end_path:out1:RootOutput(write) 0.337679 0.606665 0.885327 0.58501 0.141343 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4924.72 MB
Peak resident set size usage (VmHWM): 2342.48 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0315_dataflow7_datawriter_0_20240905T214906_reco_stage1_reco_stage2_20240926T105835_keepup.root
\tHists: np04hd_raw_run029092_0315_dataflow7_datawriter_0_20240905T214906_reco_stage1_reco_stage2_20240926T105835_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
