Jobsub ID 79140.1@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 79140.1@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3460 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:43:46 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce02 |
| Last heartbeat | 2024-09-26 15:47:51 |
| From worker node | Hostname | wn1905240.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7513 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:45:02 |
| Input files | hd-protodune:np04hd_raw_run029218_0098_dataflow0_datawriter_0_20240917T165221.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:47:51 |
| Saved logs | justin-logs:79140.1-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
..
C:0 T:1 1840 XUs and 959 XVs -> 168 XUVs
C:0 T:2 7899 XUs and 20813 XVs -> 624 XUVs
C:0 T:3 136 XUs and 222 XVs -> 3 XUVs
C:0 T:4 55 XUs and 46 XVs -> 3 XUVs
C:0 T:5 14261 XUs and 26895 XVs -> 435 XUVs
C:0 T:6 32048 XUs and 41985 XVs -> 1507 XUVs
C:0 T:7 607 XUs and 750 XVs -> 51 XUVs
2791 XUVs total
1128 collection wire objects
2791 potential space points
Neighbour search...
135249 tests to find 82154 neighbours
Iterating with no regularization...
Begin: 6.25788e+06
0 5.9029e+06
1 5.87915e+06
2 5.8756e+06
Now with regularization...
Begin: 5.6012e+06
0 5.59982e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29218 subRun: 1 event: 24544 at 26-Sep-2024 16:45:19 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 55247 traces, 55247 summary
Retagger: tagging trace set: gauss with 41859 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41859 traces tagged "gauss"
FrameSaver: q=933780 n=797703 tag=gauss
wclsFrameSaver: saving 55247 traces tagged "wiener"
FrameSaver: q=988067 n=722769 tag=wiener
2520 hits in TPC 0
1763 hits in TPC 1
4778 hits in TPC 2
4373 hits in TPC 3
1960 hits in TPC 4
3439 hits in TPC 5
6253 hits in TPC 6
3968 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 180 XUs and 183 XVs -> 3 XUVs
C:0 T:2 16298 XUs and 10302 XVs -> 1018 XUVs
C:0 T:3 671 XUs and 375 XVs -> 48 XUVs
C:0 T:4 70 XUs and 97 XVs -> 2 XUVs
C:0 T:5 5422 XUs and 4715 XVs -> 171 XUVs
C:0 T:6 13260 XUs and 16021 XVs -> 734 XUVs
C:0 T:7 88 XUs and 193 XVs -> 2 XUVs
1978 XUVs total
949 collection wire objects
1978 potential space points
Neighbour search...
36230 tests to find 21770 neighbours
Iterating with no regularization...
Begin: 6.11728e+06
0 5.88888e+06
1 5.86737e+06
2 5.86351e+06
Now with regularization...
Begin: 5.3207e+06
0 5.31957e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 31st record. run: 29218 subRun: 1 event: 24552 at 26-Sep-2024 16:46:13 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 90703 traces, 90703 summary
Retagger: tagging trace set: gauss with 64350 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 64350 traces tagged "gauss"
FrameSaver: q=1.4044e+06 n=1142174 tag=gauss
wclsFrameSaver: saving 90703 traces tagged "wiener"
FrameSaver: q=1.53356e+06 n=1024563 tag=wiener
4858 hits in TPC 0
4062 hits in TPC 1
9083 hits in TPC 2
4744 hits in TPC 3
3203 hits in TPC 4
5788 hits in TPC 5
8715 hits in TPC 6
5697 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 4856 XUs and 8952 XVs -> 1384 XUVs
C:0 T:2 50007 XUs and 44383 XVs -> 2321 XUVs
C:0 T:3 2311 XUs and 2891 XVs -> 756 XUVs
C:0 T:4 496 XUs and 858 XVs -> 50 XUVs
C:0 T:5 18007 XUs and 17403 XVs -> 1800 XUVs
C:0 T:6 50009 XUs and 37479 XVs -> 2232 XUVs
C:0 T:7 237 XUs and 248 XVs -> 10 XUVs
8553 XUVs total
2184 collection wire objects
8553 potential space points
Neighbour search...
454115 tests to find 261044 neighbours
Iterating with no regularization...
Begin: 9.46117e+06
0 8.44733e+06
1 8.33765e+06
2 8.3144e+06
3 8.30579e+06
4 8.3018e+06
Now with regularization...
Begin: 7.58745e+06
0 7.58427e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 4 event 24552 TrkackID 17 startx, y, z 101.014 601.164 47.5677 wireno, peakt tpcno 176 3901.87 2 dist 33.8173min T, max_T2693.78 3986.25
no of hits closeby 4 event 24552 TrkackID 17 startx, y, z 101.014 601.164 47.5677 wireno, peakt tpcno 176 3901.87 2 dist 33.8173min T, max_T2693.78 3986.25
26-Sep-2024 16:47:31 BST Closed output file "np04hd_raw_run029218_0098_dataflow0_datawriter_0_20240917T165221_reco_stage1_reco_stage2_20240926T154731_keepup.root"
26-Sep-2024 16:47:31 BST Closed input file "np04hd_raw_run029218_0098_dataflow0_datawriter_0_20240917T165221_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 49.6036 68.8557 118.192 66.4258 15.5911 31
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000398669 0.00271723 0.00529674 0.00283155 0.00111265 31
produce:fembfilter:PDHDFEMBFilter 0.464875 0.622968 0.803441 0.574014 0.0974753 31
produce:wclsdatahd:WireCellToolkit 26.6278 32.0851 47.1316 31.9715 3.86976 31
produce:gaushit:GausHitFinder 0.243921 0.550448 1.06743 0.561499 0.18932 31
produce:nhitsfilter:NumberOfHitsFilter 0.000198182 0.000328022 0.0011573 0.000307297 0.000165404 31
produce:reco3d:SpacePointSolver 0.0522202 0.482844 1.06523 0.452639 0.29614 31
produce:hitpdune:DisambigFromSpacePoints 0.0821712 0.340165 0.702573 0.257165 0.194687 31
produce:pandora:StandardPandora 3.09426 8.18285 26.9549 6.95687 4.67509 31
produce:pandoraWriter:StandardPandora 0.0538223 0.131929 0.231275 0.120361 0.0454316 31
produce:pandoraTrack:LArPandoraTrackCreation 0.231649 0.831529 1.68719 0.788004 0.356305 31
produce:pandoraShower:LArPandoraShowerCreation 0.241693 1.03961 4.85687 0.872844 0.821088 31
produce:pandoracalo:Calorimetry 0.130058 0.447604 1.22253 0.469039 0.203666 31
produce:pandoracalonosce:Calorimetry 0.117064 0.3923 1.05113 0.367681 0.18084 31
produce:pandorapid:Chi2ParticleID 0.000695637 0.00165406 0.00457244 0.00149348 0.000705812 31
produce:pandoraShowercalo:ShowerCalorimetry 0.216313 0.901047 3.40739 0.796051 0.596932 31
produce:pandoraShowercalonosce:ShowerCalorimetry 0.212801 0.878508 3.25756 0.75282 0.581124 31
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0470674 0.108282 0.234777 0.0955365 0.0449005 31
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0178833 0.0368595 0.0794818 0.0314519 0.0152067 31
produce:emtrkmichelid:EmTrackMichelId 11.4698 20.5731 34.4766 19.6125 5.77381 31
[art]:TriggerResults:TriggerResultInserter 1.8375e-05 5.24742e-05 0.000125536 5.0885e-05 2.60339e-05 31
end_path:out1:RootOutput 3.677e-06 1.1831e-05 4.2219e-05 9.858e-06 7.14546e-06 31
end_path:michelremoving:michelremoving 0.0130291 0.0607023 0.381723 0.0298387 0.0727353 31
end_path:michelremovinggnocchi:michelremoving 0.0111313 0.0577285 0.352697 0.02821 0.0699112 31
end_path:calibanagnocchi:CalibAnaTree 0.128515 0.326795 0.63544 0.313509 0.13554 31
end_path:out1:RootOutput(write) 0.351453 0.670099 1.24674 0.640856 0.211257 31
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4932.79 MB
Peak resident set size usage (VmHWM): 2560.38 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029218_0098_dataflow0_datawriter_0_20240917T165221_reco_stage1_reco_stage2_20240926T154731_keepup.root
\tHists: np04hd_raw_run029218_0098_dataflow0_datawriter_0_20240917T165221_reco_stage1_reco_stage2_20240926T154731_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
