Jobsub ID 263757.92@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263757.92@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:33 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce3 |
| Last heartbeat | 2024-09-26 14:55:37 |
| From worker node | Hostname | n259.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:29 |
| Input files | hd-protodune:np04hd_raw_run029094_0161_dataflow5_datawriter_0_20240906T095206.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:55:37 |
| Saved logs | justin-logs:263757.92-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
.22787e+06
0 6.21425e+06
1 6.12775e+06
2 6.11266e+06
3 6.10773e+06
Now with regularization...
Begin: 5.70756e+06
0 5.70386e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29094 subRun: 1 event: 38885 at 26-Sep-2024 15:52:25 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 72616 traces, 72616 summary
Retagger: tagging trace set: gauss with 56149 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56149 traces tagged "gauss"
FrameSaver: q=1.23811e+06 n=1095126 tag=gauss
wclsFrameSaver: saving 72616 traces tagged "wiener"
FrameSaver: q=1.325e+06 n=1004061 tag=wiener
2863 hits in TPC 0
2217 hits in TPC 1
5414 hits in TPC 2
4751 hits in TPC 3
4837 hits in TPC 4
9039 hits in TPC 5
8353 hits in TPC 6
4443 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 212 XUs and 44 XVs -> 2 XUVs
C:0 T:2 19475 XUs and 15563 XVs -> 687 XUVs
C:0 T:3 133 XUs and 246 XVs -> 11 XUVs
C:0 T:4 67 XUs and 93 XVs -> 3 XUVs
C:0 T:5 34410 XUs and 31779 XVs -> 724 XUVs
C:0 T:6 20062 XUs and 41311 XVs -> 1121 XUVs
C:0 T:7 420 XUs and 673 XVs -> 10 XUVs
2558 XUVs total
1696 collection wire objects
2558 potential space points
Neighbour search...
52384 tests to find 25784 neighbours
Iterating with no regularization...
Begin: 1.27609e+07
0 1.16917e+07
1 1.16109e+07
2 1.1607e+07
Now with regularization...
Begin: 1.0871e+07
0 1.08675e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 5 event 38885 TrkackID 0 startx, y, z -198.97 315.701 459.197 wireno, peakt tpcno 189 3243.15 2 dist 2.12613min T, max_T1859.15 4399.61
no of hits closeby 6 event 38885 TrkackID 2 startx, y, z -47.9306 594.689 134.951 wireno, peakt tpcno 475 2150.28 6 dist 3.33647min T, max_T2139.93 3805.78
no of hits closeby 5 event 38885 TrkackID 0 startx, y, z -198.97 315.701 459.197 wireno, peakt tpcno 189 3243.15 2 dist 2.12613min T, max_T1859.15 4399.61
no of hits closeby 6 event 38885 TrkackID 2 startx, y, z -47.9306 594.689 134.951 wireno, peakt tpcno 475 2150.28 6 dist 3.33647min T, max_T2139.93 3805.78
Begin processing the 30th record. run: 29094 subRun: 1 event: 38893 at 26-Sep-2024 15:54:03 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 41923 traces, 41923 summary
Retagger: tagging trace set: gauss with 30781 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 30781 traces tagged "gauss"
FrameSaver: q=715456 n=595068 tag=gauss
wclsFrameSaver: saving 41923 traces tagged "wiener"
FrameSaver: q=758125 n=539638 tag=wiener
1390 hits in TPC 0
1396 hits in TPC 1
4059 hits in TPC 2
2359 hits in TPC 3
1757 hits in TPC 4
3763 hits in TPC 5
3926 hits in TPC 6
2929 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 61 XUs and 48 XVs -> 2 XUVs
C:0 T:2 4938 XUs and 9268 XVs -> 585 XUVs
C:0 T:3 515 XUs and 693 XVs -> 34 XUVs
C:0 T:4 122 XUs and 223 XVs -> 7 XUVs
C:0 T:5 4787 XUs and 10914 XVs -> 438 XUVs
C:0 T:6 4259 XUs and 12082 XVs -> 617 XUVs
C:0 T:7 173 XUs and 221 XVs -> 16 XUVs
1699 XUVs total
1030 collection wire objects
1699 potential space points
Neighbour search...
28971 tests to find 16528 neighbours
Iterating with no regularization...
Begin: 6.05993e+06
0 5.72278e+06
1 5.7073e+06
2 5.70567e+06
Now with regularization...
Begin: 5.26251e+06
0 5.26203e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 38893 TrkackID 12 startx, y, z 51.7531 308.868 47.9178 wireno, peakt tpcno 103 630 2 dist 76.5997min T, max_T5 651.34
no of hits closeby 0 event 38893 TrkackID 12 startx, y, z 51.7531 308.868 47.9178 wireno, peakt tpcno 103 630 2 dist 76.5997min T, max_T5 651.34
26-Sep-2024 15:55:04 BST Closed output file "np04hd_raw_run029094_0161_dataflow5_datawriter_0_20240906T095206_reco_stage1_reco_stage2_20240926T145504_keepup.root"
26-Sep-2024 15:55:04 BST Closed input file "np04hd_raw_run029094_0161_dataflow5_datawriter_0_20240906T095206_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 54.1089 79.3898 106.549 79.7801 14.0351 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000942647 0.00242842 0.00372053 0.00246603 0.000761226 30
produce:fembfilter:PDHDFEMBFilter 0.628163 0.638685 0.665887 0.639148 0.00774177 30
produce:wclsdatahd:WireCellToolkit 33.5073 34.7523 36.2463 34.6744 0.714619 30
produce:gaushit:GausHitFinder 0.386874 0.702808 1.02542 0.688044 0.179357 30
produce:nhitsfilter:NumberOfHitsFilter 0.000228283 0.00032725 0.000452949 0.000331809 5.8851e-05 30
produce:reco3d:SpacePointSolver 0.114804 0.63029 1.45525 0.620839 0.343212 30
produce:hitpdune:DisambigFromSpacePoints 0.138935 0.536551 1.19565 0.513272 0.251381 30
produce:pandora:StandardPandora 3.54513 10.6636 20.5648 10.156 4.5443 30
produce:pandoraWriter:StandardPandora 0.0719778 0.140202 0.197417 0.142284 0.0358048 30
produce:pandoraTrack:LArPandoraTrackCreation 0.310275 0.978802 1.96636 0.96339 0.445028 30
produce:pandoraShower:LArPandoraShowerCreation 0.304584 1.13102 2.07693 1.08007 0.553213 30
produce:pandoracalo:Calorimetry 0.19421 0.545892 0.895527 0.547159 0.200587 30
produce:pandoracalonosce:Calorimetry 0.15867 0.477556 0.815964 0.490357 0.184916 30
produce:pandorapid:Chi2ParticleID 0.00121259 0.00194904 0.00429465 0.00193424 0.000550868 30
produce:pandoraShowercalo:ShowerCalorimetry 0.267148 1.06384 1.9051 1.06095 0.518732 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.260404 1.02282 1.87154 1.00482 0.498383 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0750925 0.14112 0.196384 0.133516 0.0340462 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0233418 0.0481884 0.0650428 0.0480242 0.0115909 30
produce:emtrkmichelid:EmTrackMichelId 13.105 24.504 34.3257 25.2317 5.62909 30
[art]:TriggerResults:TriggerResultInserter 2.4166e-05 3.25072e-05 6.2018e-05 3.00825e-05 7.86493e-06 30
end_path:out1:RootOutput 9.338e-06 1.15288e-05 2.4807e-05 1.0765e-05 2.75809e-06 30
end_path:michelremoving:michelremoving 0.0158697 0.079072 0.328385 0.0318019 0.0817384 30
end_path:michelremovinggnocchi:michelremoving 0.0129971 0.0767271 0.32513 0.0283406 0.0820518 30
end_path:calibanagnocchi:CalibAnaTree 0.121013 0.372325 0.686441 0.360673 0.160006 30
end_path:out1:RootOutput(write) 0.383947 0.752293 1.04007 0.770503 0.181019 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4924.53 MB
Peak resident set size usage (VmHWM): 2563.64 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0161_dataflow5_datawriter_0_20240906T095206_reco_stage1_reco_stage2_20240926T145504_keepup.root
\tHists: np04hd_raw_run029094_0161_dataflow5_datawriter_0_20240906T095206_reco_stage1_reco_stage2_20240926T145504_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
