Jobsub ID 263757.97@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263757.97@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:33 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce3 |
| Last heartbeat | 2024-09-26 14:50:03 |
| From worker node | Hostname | n244.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:17 |
| Input files | hd-protodune:np04hd_raw_run029094_0159_dataflow3_datawriter_0_20240906T095106.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:50:03 |
| Saved logs | justin-logs:263757.97-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
cooked to 5859 ticks
wclsFrameSaver: saving 36611 traces tagged "gauss"
FrameSaver: q=801299 n=740558 tag=gauss
wclsFrameSaver: saving 48621 traces tagged "wiener"
FrameSaver: q=849742 n=675857 tag=wiener
4659 hits in TPC 0
3300 hits in TPC 1
3289 hits in TPC 2
2305 hits in TPC 3
2341 hits in TPC 4
5401 hits in TPC 5
3526 hits in TPC 6
1997 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3272 XUs and 4676 XVs -> 1320 XUVs
C:0 T:2 8733 XUs and 4804 XVs -> 162 XUVs
C:0 T:3 53 XUs and 25 XVs -> 1 XUVs
C:0 T:4 24 XUs and 15 XVs -> 0 XUVs
C:0 T:5 9963 XUs and 17441 XVs -> 267 XUVs
C:0 T:6 7638 XUs and 4674 XVs -> 367 XUVs
C:0 T:7 222 XUs and 269 XVs -> 5 XUVs
2122 XUVs total
759 collection wire objects
2122 potential space points
Neighbour search...
129634 tests to find 81936 neighbours
Iterating with no regularization...
Begin: 3.68071e+06
0 3.58123e+06
1 3.57647e+06
2 3.57624e+06
Now with regularization...
Begin: 3.42121e+06
0 3.42112e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 38387 TrkackID 1 startx, y, z 199.551 606.024 210.58 wireno, peakt tpcno 332 3253.05 2 dist 409.52min T, max_T8.87402 3286.64
no of hits closeby 0 event 38387 TrkackID 16 startx, y, z 325.585 128.727 462.331 wireno, peakt tpcno 260 3534.58 7 dist 1.18107min T, max_T3455 3856.35
no of hits closeby 1 event 38387 TrkackID 19 startx, y, z 38.4176 585.08 436.503 wireno, peakt tpcno 448 1226.27 5 dist 0min T, max_T910.315 1378.52
no of hits closeby 0 event 38387 TrkackID 1 startx, y, z 199.551 606.024 210.58 wireno, peakt tpcno 332 3253.05 2 dist 409.52min T, max_T8.87402 3286.64
no of hits closeby 0 event 38387 TrkackID 16 startx, y, z 325.585 128.727 462.331 wireno, peakt tpcno 260 3534.58 7 dist 1.18107min T, max_T3455 3856.35
no of hits closeby 1 event 38387 TrkackID 19 startx, y, z 38.4176 585.08 436.503 wireno, peakt tpcno 448 1226.27 5 dist 0min T, max_T910.315 1378.52
Begin processing the 30th record. run: 29094 subRun: 1 event: 38395 at 26-Sep-2024 15:48:12 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67751 traces, 67751 summary
Retagger: tagging trace set: gauss with 49793 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49793 traces tagged "gauss"
FrameSaver: q=1.09092e+06 n=942008 tag=gauss
wclsFrameSaver: saving 67751 traces tagged "wiener"
FrameSaver: q=1.16895e+06 n=853121 tag=wiener
2959 hits in TPC 0
2226 hits in TPC 1
8129 hits in TPC 2
6477 hits in TPC 3
2800 hits in TPC 4
5779 hits in TPC 5
5429 hits in TPC 6
2745 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 406 XUs and 282 XVs -> 16 XUVs
C:0 T:2 36313 XUs and 29441 XVs -> 824 XUVs
C:0 T:3 615 XUs and 545 XVs -> 6 XUVs
C:0 T:4 121 XUs and 123 XVs -> 3 XUVs
C:0 T:5 16215 XUs and 20024 XVs -> 654 XUVs
C:0 T:6 26413 XUs and 18109 XVs -> 1244 XUVs
C:0 T:7 227 XUs and 260 XVs -> 19 XUVs
2766 XUVs total
1280 collection wire objects
2766 potential space points
Neighbour search...
118312 tests to find 71112 neighbours
Iterating with no regularization...
Begin: 7.28428e+06
0 6.90124e+06
1 6.87977e+06
2 6.87669e+06
Now with regularization...
Begin: 6.50428e+06
0 6.5027e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 38395 TrkackID 2 startx, y, z 107.425 595.145 400.919 wireno, peakt tpcno 0 848.604 5 dist 132.743min T, max_T6.73415 1645
no of hits closeby 0 event 38395 TrkackID 5 startx, y, z 193.622 606.329 9.44641 wireno, peakt tpcno 225 4689.33 2 dist 197.18min T, max_T2268.69 4735.12
no of hits closeby 0 event 38395 TrkackID 2 startx, y, z 107.425 595.145 400.919 wireno, peakt tpcno 0 848.604 5 dist 132.743min T, max_T6.73415 1645
no of hits closeby 0 event 38395 TrkackID 5 startx, y, z 193.622 606.329 9.44641 wireno, peakt tpcno 225 4689.33 2 dist 197.18min T, max_T2268.69 4735.12
26-Sep-2024 15:49:33 BST Closed output file "np04hd_raw_run029094_0159_dataflow3_datawriter_0_20240906T095106_reco_stage1_reco_stage2_20240926T144933_keepup.root"
26-Sep-2024 15:49:33 BST Closed input file "np04hd_raw_run029094_0159_dataflow3_datawriter_0_20240906T095106_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 51.3899 70.0472 105.186 68.0225 11.2217 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000386441 0.00214935 0.00622991 0.00194892 0.00115802 30
produce:fembfilter:PDHDFEMBFilter 0.447287 0.624119 0.720555 0.661235 0.0913208 30
produce:wclsdatahd:WireCellToolkit 25.361 33.8422 37.2866 34.9549 3.19831 30
produce:gaushit:GausHitFinder 0.292253 0.591498 1.00546 0.575744 0.159869 30
produce:nhitsfilter:NumberOfHitsFilter 0.000179629 0.000289803 0.000432126 0.000276637 5.82779e-05 30
produce:reco3d:SpacePointSolver 0.0661502 0.547991 3.22468 0.413572 0.609183 30
produce:hitpdune:DisambigFromSpacePoints 0.0836598 0.402509 1.1441 0.337265 0.223641 30
produce:pandora:StandardPandora 4.74536 8.7181 18.4989 7.44158 3.278 30
produce:pandoraWriter:StandardPandora 0.0632146 0.125977 0.231373 0.128813 0.0375062 30
produce:pandoraTrack:LArPandoraTrackCreation 0.300345 0.765862 1.77629 0.70597 0.325299 30
produce:pandoraShower:LArPandoraShowerCreation 0.251299 0.879135 2.34721 0.812897 0.417911 30
produce:pandoracalo:Calorimetry 0.182738 0.446725 0.89551 0.430001 0.157317 30
produce:pandoracalonosce:Calorimetry 0.157696 0.392955 0.862708 0.379967 0.15253 30
produce:pandorapid:Chi2ParticleID 0.000913216 0.00171619 0.00358208 0.00166897 0.000532154 30
produce:pandoraShowercalo:ShowerCalorimetry 0.230801 0.823499 2.01616 0.788836 0.386367 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.224886 0.797369 1.9666 0.776303 0.375133 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0580488 0.120996 0.203495 0.121993 0.041938 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0194754 0.0403593 0.0665139 0.0399396 0.013866 30
produce:emtrkmichelid:EmTrackMichelId 10.7196 19.72 32.7272 19.3122 4.05511 30
[art]:TriggerResults:TriggerResultInserter 1.7563e-05 3.34644e-05 7.3058e-05 3.1309e-05 1.16732e-05 30
end_path:out1:RootOutput 6.953e-06 1.88734e-05 0.000244843 1.06705e-05 4.20762e-05 30
end_path:michelremoving:michelremoving 0.0113897 0.0751214 0.277196 0.0268015 0.0761181 30
end_path:michelremovinggnocchi:michelremoving 0.0106844 0.0731001 0.276391 0.023201 0.0758893 30
end_path:calibanagnocchi:CalibAnaTree 0.1064 0.312108 0.522012 0.298786 0.107638 30
end_path:out1:RootOutput(write) 0.363293 0.645663 1.04265 0.637224 0.163825 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4927.1 MB
Peak resident set size usage (VmHWM): 2593.7 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0159_dataflow3_datawriter_0_20240906T095106_reco_stage1_reco_stage2_20240926T144933_keepup.root
\tHists: np04hd_raw_run029094_0159_dataflow3_datawriter_0_20240906T095106_reco_stage1_reco_stage2_20240926T144933_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
