Jobsub ID 263681.40@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263681.40@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:07:21 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:52:34 |
| From worker node | Hostname | n257.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:08:10 |
| Input files | hd-protodune:np04hd_raw_run029107_2164_dataflow5_datawriter_0_20240908T024310.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:52:34 |
| Saved logs | justin-logs:263681.40-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
-----------------------------------------------------------------------------
no of hits closeby 0 event 519605 TrkackID 2 startx, y, z 148.567 597.235 314.858 wireno, peakt tpcno 357 9.17324 5 dist 0min T, max_T9.17324 2035
no of hits closeby 0 event 519605 TrkackID 11 startx, y, z 90.8988 524.657 34.9318 wireno, peakt tpcno 125 1087.63 2 dist 124.052min T, max_T3.24481 1124.04
no of hits closeby 0 event 519605 TrkackID 2 startx, y, z 148.567 597.235 314.858 wireno, peakt tpcno 357 9.17324 5 dist 0min T, max_T9.17324 2035
no of hits closeby 0 event 519605 TrkackID 11 startx, y, z 90.8988 524.657 34.9318 wireno, peakt tpcno 125 1087.63 2 dist 124.052min T, max_T3.24481 1124.04
Begin processing the 29th record. run: 29107 subRun: 1 event: 519613 at 26-Sep-2024 15:47:54 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 49784 traces, 49784 summary
Retagger: tagging trace set: gauss with 37577 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 37577 traces tagged "gauss"
FrameSaver: q=917843 n=716330 tag=gauss
wclsFrameSaver: saving 49784 traces tagged "wiener"
FrameSaver: q=982899 n=653210 tag=wiener
579 hits in TPC 0
1432 hits in TPC 1
7723 hits in TPC 2
4029 hits in TPC 3
1227 hits in TPC 4
3049 hits in TPC 5
5860 hits in TPC 6
2968 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1014 XUs and 1151 XVs -> 124 XUVs
C:0 T:2 68692 XUs and 52108 XVs -> 1553 XUVs
C:0 T:3 517 XUs and 529 XVs -> 14 XUVs
C:0 T:4 21 XUs and 26 XVs -> 0 XUVs
C:0 T:5 3638 XUs and 3022 XVs -> 190 XUVs
C:0 T:6 19460 XUs and 19070 XVs -> 370 XUVs
C:0 T:7 21 XUs and 43 XVs -> 0 XUVs
2251 XUVs total
916 collection wire objects
2251 potential space points
Neighbour search...
69355 tests to find 36662 neighbours
Iterating with no regularization...
Begin: 1.33886e+07
0 1.25812e+07
1 1.25571e+07
2 1.25544e+07
Now with regularization...
Begin: 1.19646e+07
0 1.19614e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 519621 at 26-Sep-2024 15:49:21 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 97733 traces, 97733 summary
Retagger: tagging trace set: gauss with 71828 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 71828 traces tagged "gauss"
FrameSaver: q=1.65096e+06 n=1372512 tag=gauss
wclsFrameSaver: saving 97733 traces tagged "wiener"
FrameSaver: q=1.78328e+06 n=1250812 tag=wiener
5889 hits in TPC 0
4343 hits in TPC 1
7209 hits in TPC 2
3932 hits in TPC 3
5830 hits in TPC 4
10762 hits in TPC 5
8087 hits in TPC 6
7141 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1426 XUs and 977 XVs -> 34 XUVs
C:0 T:2 56477 XUs and 15648 XVs -> 642 XUVs
C:0 T:3 224 XUs and 212 XVs -> 5 XUVs
C:0 T:4 260 XUs and 457 XVs -> 11 XUVs
C:0 T:5 34989 XUs and 85980 XVs -> 2569 XUVs
C:0 T:6 93877 XUs and 30456 XVs -> 1186 XUVs
C:0 T:7 858 XUs and 286 XVs -> 18 XUVs
4465 XUVs total
2396 collection wire objects
4465 potential space points
Neighbour search...
124589 tests to find 66296 neighbours
Iterating with no regularization...
Begin: 4.33774e+07
0 3.83052e+07
1 3.7902e+07
2 3.78879e+07
Now with regularization...
Begin: 3.72025e+07
0 3.71985e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 4 event 519621 TrkackID 19 startx, y, z -27.8801 587.755 283.569 wireno, peakt tpcno 327 7.06001 2 dist 0min T, max_T1.57155 912.499
no of hits closeby 5 event 519621 TrkackID 30 startx, y, z -52.6136 211.939 179.642 wireno, peakt tpcno 435 515.178 2 dist 2.52892min T, max_T6.04179 679.458
no of hits closeby 4 event 519621 TrkackID 19 startx, y, z -27.8801 587.755 283.569 wireno, peakt tpcno 327 7.06001 2 dist 0min T, max_T1.57155 912.499
no of hits closeby 5 event 519621 TrkackID 30 startx, y, z -52.6136 211.939 179.642 wireno, peakt tpcno 435 515.178 2 dist 2.52892min T, max_T6.04179 679.458
26-Sep-2024 15:51:57 BST Closed output file "np04hd_raw_run029107_2164_dataflow5_datawriter_0_20240908T024310_reco_stage1_reco_stage2_20240926T145157_keepup.root"
26-Sep-2024 15:51:57 BST Closed input file "np04hd_raw_run029107_2164_dataflow5_datawriter_0_20240908T024310_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 69.3886 107.291 167.352 99.5495 24.1908 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000536339 0.00285724 0.00577679 0.00282293 0.00130166 30
produce:fembfilter:PDHDFEMBFilter 0.781003 0.794573 0.823766 0.790999 0.0115838 30
produce:wclsdatahd:WireCellToolkit 41.5168 43.0316 45.0666 42.9458 0.930311 30
produce:gaushit:GausHitFinder 0.42345 0.951977 1.5555 0.956477 0.252461 30
produce:nhitsfilter:NumberOfHitsFilter 0.000263851 0.000446834 0.00112176 0.000425388 0.000168465 30
produce:reco3d:SpacePointSolver 0.204987 1.12952 3.70137 0.866918 0.88894 30
produce:hitpdune:DisambigFromSpacePoints 0.233987 0.777322 1.88915 0.68323 0.42625 30
produce:pandora:StandardPandora 6.42397 17.2531 46.4098 13.3319 9.99929 30
produce:pandoraWriter:StandardPandora 0.0952239 0.197158 0.292279 0.192803 0.0498779 30
produce:pandoraTrack:LArPandoraTrackCreation 0.394238 1.59187 3.12389 1.46405 0.683895 30
produce:pandoraShower:LArPandoraShowerCreation 0.375548 1.95011 5.08576 1.78202 1.1254 30
produce:pandoracalo:Calorimetry 0.247243 0.829842 1.54024 0.788132 0.320476 30
produce:pandoracalonosce:Calorimetry 0.204207 0.74983 1.41599 0.706154 0.301035 30
produce:pandorapid:Chi2ParticleID 0.00135513 0.00264607 0.00477744 0.00249251 0.000681816 30
produce:pandoraShowercalo:ShowerCalorimetry 0.322636 1.73174 4.68526 1.54501 1.00411 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.319982 1.68024 4.84389 1.52265 1.00592 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.103935 0.188444 0.305864 0.180163 0.0462317 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0358308 0.063803 0.094784 0.062357 0.0143339 30
produce:emtrkmichelid:EmTrackMichelId 16.5008 32.398 48.8164 31.6702 8.04649 30
[art]:TriggerResults:TriggerResultInserter 2.7242e-05 4.95699e-05 0.000127883 4.4344e-05 2.31904e-05 30
end_path:out1:RootOutput 1.1692e-05 2.15223e-05 0.000217633 1.3776e-05 3.66489e-05 30
end_path:michelremoving:michelremoving 0.017276 0.113161 0.496005 0.0425859 0.127446 30
end_path:michelremovinggnocchi:michelremoving 0.0162018 0.109857 0.486214 0.0395805 0.123442 30
end_path:calibanagnocchi:CalibAnaTree 0.135214 0.585496 1.4884 0.526231 0.271248 30
end_path:out1:RootOutput(write) 0.521117 0.994561 1.52386 0.95245 0.249181 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4948.37 MB
Peak resident set size usage (VmHWM): 2608.56 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2164_dataflow5_datawriter_0_20240908T024310_reco_stage1_reco_stage2_20240926T145157_keepup.root
\tHists: np04hd_raw_run029107_2164_dataflow5_datawriter_0_20240908T024310_reco_stage1_reco_stage2_20240926T145157_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
