Jobsub ID 263758.19@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263758.19@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:34 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:51:43 |
| From worker node | Hostname | n258.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:01 |
| Input files | hd-protodune:np04hd_raw_run029107_0847_dataflow5_datawriter_0_20240907T043417.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:51:43 |
| Saved logs | justin-logs:263758.19-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
in TPC 2
1911 hits in TPC 3
2853 hits in TPC 4
5165 hits in TPC 5
4795 hits in TPC 6
3025 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 2 XUs and 1 XVs -> 0 XUVs
C:0 T:1 5689 XUs and 5264 XVs -> 189 XUVs
C:0 T:2 6966 XUs and 7215 XVs -> 134 XUVs
C:0 T:3 17 XUs and 18 XVs -> 0 XUVs
C:0 T:4 82 XUs and 160 XVs -> 3 XUVs
C:0 T:5 11581 XUs and 20408 XVs -> 396 XUVs
C:0 T:6 13096 XUs and 14917 XVs -> 258 XUVs
C:0 T:7 337 XUs and 2063 XVs -> 10 XUVs
990 XUVs total
678 collection wire objects
990 potential space points
Neighbour search...
13578 tests to find 8424 neighbours
Iterating with no regularization...
Begin: 2.07768e+06
0 2.01304e+06
1 2.01099e+06
2 2.01072e+06
Now with regularization...
Begin: 1.86975e+06
0 1.86967e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 203533 at 26-Sep-2024 15:48:26 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 85831 traces, 85831 summary
Retagger: tagging trace set: gauss with 61544 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 61544 traces tagged "gauss"
FrameSaver: q=1.37453e+06 n=1095292 tag=gauss
wclsFrameSaver: saving 85831 traces tagged "wiener"
FrameSaver: q=1.48459e+06 n=975106 tag=wiener
3380 hits in TPC 0
4167 hits in TPC 1
10823 hits in TPC 2
5517 hits in TPC 3
3062 hits in TPC 4
6261 hits in TPC 5
6119 hits in TPC 6
4163 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1253 XUs and 2376 XVs -> 69 XUVs
C:0 T:2 116956 XUs and 70149 XVs -> 1898 XUVs
C:0 T:3 1526 XUs and 780 XVs -> 35 XUVs
C:0 T:5 10169 XUs and 32834 XVs -> 1387 XUVs
C:0 T:6 48361 XUs and 24376 XVs -> 934 XUVs
C:0 T:7 55 XUs and 39 XVs -> 2 XUVs
4325 XUVs total
2226 collection wire objects
4325 potential space points
Neighbour search...
100959 tests to find 56134 neighbours
Iterating with no regularization...
Begin: 2.02626e+07
0 1.92157e+07
1 1.91194e+07
2 1.91151e+07
Now with regularization...
Begin: 1.84008e+07
0 1.83994e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 203541 at 26-Sep-2024 15:49:57 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 71322 traces, 71322 summary
Retagger: tagging trace set: gauss with 50376 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50376 traces tagged "gauss"
FrameSaver: q=1.13319e+06 n=915201 tag=gauss
wclsFrameSaver: saving 71322 traces tagged "wiener"
FrameSaver: q=1.21232e+06 n=818975 tag=wiener
2377 hits in TPC 0
1693 hits in TPC 1
6798 hits in TPC 2
2496 hits in TPC 3
5257 hits in TPC 4
7098 hits in TPC 5
6391 hits in TPC 6
3532 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 565 XUs and 540 XVs -> 6 XUVs
C:0 T:2 14277 XUs and 15579 XVs -> 308 XUVs
C:0 T:3 214 XUs and 282 XVs -> 1 XUVs
C:0 T:4 279 XUs and 446 XVs -> 27 XUVs
C:0 T:5 21093 XUs and 35179 XVs -> 1343 XUVs
C:0 T:6 15939 XUs and 22560 XVs -> 610 XUVs
C:0 T:7 391 XUs and 244 XVs -> 13 XUVs
2308 XUVs total
1388 collection wire objects
2308 potential space points
Neighbour search...
29118 tests to find 17436 neighbours
Iterating with no regularization...
Begin: 8.3813e+06
0 7.95348e+06
1 7.93497e+06
2 7.93396e+06
Now with regularization...
Begin: 7.35275e+06
0 7.35229e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 203541 TrkackID 9 startx, y, z -64.3599 551.272 453.372 wireno, peakt tpcno 426 4958.44 6 dist 0min T, max_T4249.29 5853.07
no of hits closeby 0 event 203541 TrkackID 9 startx, y, z -64.3599 551.272 453.372 wireno, peakt tpcno 426 4958.44 6 dist 0min T, max_T4249.29 5853.07
26-Sep-2024 15:51:14 BST Closed output file "np04hd_raw_run029107_0847_dataflow5_datawriter_0_20240907T043417_reco_stage1_reco_stage2_20240926T145114_keepup.root"
26-Sep-2024 15:51:14 BST Closed input file "np04hd_raw_run029107_0847_dataflow5_datawriter_0_20240907T043417_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 54.7038 78.9465 144.202 74.0035 18.1041 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000345401 0.00161568 0.00292049 0.00178235 0.000701183 30
produce:fembfilter:PDHDFEMBFilter 0.518071 0.565852 0.791155 0.534782 0.0844365 30
produce:wclsdatahd:WireCellToolkit 28.404 31.3636 41.7992 30.0811 3.73162 30
produce:gaushit:GausHitFinder 0.366551 0.59117 1.11321 0.559409 0.173307 30
produce:nhitsfilter:NumberOfHitsFilter 0.000246835 0.000330635 0.000862906 0.00030463 0.000116067 30
produce:reco3d:SpacePointSolver 0.156619 0.770724 1.87279 0.626015 0.48369 30
produce:hitpdune:DisambigFromSpacePoints 0.181948 0.438857 1.2448 0.373111 0.246894 30
produce:pandora:StandardPandora 4.63604 12.253 65.7713 9.32006 10.926 30
produce:pandoraWriter:StandardPandora 0.0837889 0.127965 0.206046 0.12232 0.0329084 30
produce:pandoraTrack:LArPandoraTrackCreation 0.509092 0.920231 1.75175 0.798566 0.322556 30
produce:pandoraShower:LArPandoraShowerCreation 0.460157 1.13844 2.49647 1.09508 0.516872 30
produce:pandoracalo:Calorimetry 0.304759 0.509536 0.974788 0.463385 0.173415 30
produce:pandoracalonosce:Calorimetry 0.275306 0.462051 0.890981 0.415056 0.16191 30
produce:pandorapid:Chi2ParticleID 0.00121454 0.00166589 0.00333866 0.0015005 0.000444974 30
produce:pandoraShowercalo:ShowerCalorimetry 0.366336 1.12275 2.68515 1.058 0.581363 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.359357 1.09158 2.60312 1.02402 0.557195 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0713059 0.117857 0.223869 0.111404 0.0328595 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0250055 0.04201 0.0955477 0.0392392 0.0136563 30
produce:emtrkmichelid:EmTrackMichelId 16.2197 26.0149 39.5955 25.1225 6.78847 30
[art]:TriggerResults:TriggerResultInserter 1.558e-05 2.31065e-05 0.000105408 1.9016e-05 1.58425e-05 30
end_path:out1:RootOutput 9.027e-06 1.1852e-05 3.726e-05 1.098e-05 4.96993e-06 30
end_path:michelremoving:michelremoving 0.0145835 0.0866763 0.3681 0.0802316 0.0752756 30
end_path:michelremovinggnocchi:michelremoving 0.0131024 0.0831816 0.364677 0.0776791 0.0749365 30
end_path:calibanagnocchi:CalibAnaTree 0.209611 0.407852 0.845276 0.369154 0.15473 30
end_path:out1:RootOutput(write) 0.46775 0.734653 1.41926 0.695017 0.220934 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4996.49 MB
Peak resident set size usage (VmHWM): 2620.33 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0847_dataflow5_datawriter_0_20240907T043417_reco_stage1_reco_stage2_20240926T145114_keepup.root
\tHists: np04hd_raw_run029107_0847_dataflow5_datawriter_0_20240907T043417_reco_stage1_reco_stage2_20240926T145114_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
