Jobsub ID 263763.123@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263763.123@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:49 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 14:31:12 |
| From worker node | Hostname | wn035.hep |
| cpuinfo | AMD EPYC 7352 24-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:26:47 |
| Input files | hd-protodune:np04hd_raw_run029097_0034_dataflow7_datawriter_0_20240906T105948.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:31:12 |
| Saved logs | justin-logs:263763.123-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
XUs and 1026 XVs -> 55 XUVs
C:0 T:2 29471 XUs and 45588 XVs -> 2154 XUVs
C:0 T:3 218 XUs and 268 XVs -> 20 XUVs
C:0 T:4 447 XUs and 857 XVs -> 32 XUVs
C:0 T:5 73652 XUs and 70885 XVs -> 3533 XUVs
C:0 T:6 24585 XUs and 35615 XVs -> 1029 XUVs
C:0 T:7 158 XUs and 443 XVs -> 17 XUVs
6840 XUVs total
1901 collection wire objects
6840 potential space points
Neighbour search...
735302 tests to find 343226 neighbours
Iterating with no regularization...
Begin: 1.09136e+07
0 7.61835e+06
1 7.29799e+06
2 7.26946e+06
3 7.26047e+06
4 7.25629e+06
Now with regularization...
Begin: 6.75023e+06
0 6.74582e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 8415 at 26-Sep-2024 15:25:24 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 64292 traces, 64292 summary
Retagger: tagging trace set: gauss with 47994 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47994 traces tagged "gauss"
FrameSaver: q=1.02611e+06 n=909035 tag=gauss
wclsFrameSaver: saving 64292 traces tagged "wiener"
FrameSaver: q=1.12258e+06 n=834786 tag=wiener
4594 hits in TPC 0
3102 hits in TPC 1
6318 hits in TPC 2
2398 hits in TPC 3
4994 hits in TPC 4
7990 hits in TPC 5
4381 hits in TPC 6
1180 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 6044 XUs and 14475 XVs -> 1265 XUVs
C:0 T:2 20747 XUs and 25910 XVs -> 628 XUVs
C:0 T:3 105 XUs and 71 XVs -> 2 XUVs
C:0 T:4 526 XUs and 507 XVs -> 47 XUVs
C:0 T:5 44210 XUs and 50351 XVs -> 5552 XUVs
C:0 T:6 12987 XUs and 13481 XVs -> 241 XUVs
7735 XUVs total
1438 collection wire objects
7735 potential space points
Neighbour search...
891711 tests to find 457480 neighbours
Iterating with no regularization...
Begin: 6.81601e+06
0 5.69376e+06
1 5.58123e+06
2 5.55386e+06
3 5.54465e+06
4 5.53966e+06
Now with regularization...
Begin: 5.13389e+06
0 5.12968e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 8423 at 26-Sep-2024 15:26:32 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 73585 traces, 73585 summary
Retagger: tagging trace set: gauss with 54310 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54310 traces tagged "gauss"
FrameSaver: q=1.19294e+06 n=1049011 tag=gauss
wclsFrameSaver: saving 73585 traces tagged "wiener"
FrameSaver: q=1.29571e+06 n=952546 tag=wiener
5712 hits in TPC 0
4166 hits in TPC 1
8658 hits in TPC 2
2428 hits in TPC 3
3846 hits in TPC 4
6858 hits in TPC 5
6164 hits in TPC 6
3336 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 213 XUs and 276 XVs -> 8 XUVs
C:0 T:1 405 XUs and 529 XVs -> 31 XUVs
C:0 T:2 110360 XUs and 111525 XVs -> 7551 XUVs
C:0 T:3 846 XUs and 555 XVs -> 24 XUVs
C:0 T:4 523 XUs and 590 XVs -> 19 XUVs
C:0 T:5 15133 XUs and 26778 XVs -> 1319 XUVs
C:0 T:6 58017 XUs and 62707 XVs -> 1087 XUVs
C:0 T:7 418 XUs and 623 XVs -> 6 XUVs
10045 XUVs total
1914 collection wire objects
10045 potential space points
Neighbour search...
1080913 tests to find 435942 neighbours
Iterating with no regularization...
Begin: 1.14557e+07
0 1.00865e+07
1 1.0007e+07
2 9.99271e+06
3 9.98748e+06
Now with regularization...
Begin: 9.5513e+06
0 9.54837e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 8423 TrkackID 6 startx, y, z 87.0868 257.006 450.58 wireno, peakt tpcno 0 3565.73 5 dist 124.385min T, max_T2734.7 4536.39
no of hits closeby 0 event 8423 TrkackID 6 startx, y, z 87.0868 257.006 450.58 wireno, peakt tpcno 0 3565.73 5 dist 124.385min T, max_T2734.7 4536.39
26-Sep-2024 15:27:47 BST Closed output file "np04hd_raw_run029097_0034_dataflow7_datawriter_0_20240906T105948_reco_stage1_reco_stage2_20240926T142747_keepup.root"
26-Sep-2024 15:27:47 BST Closed input file "np04hd_raw_run029097_0034_dataflow7_datawriter_0_20240906T105948_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 56.3277 73.5704 113.887 69.0588 13.4847 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000359145 0.0015434 0.00317984 0.00155675 0.000702865 30
produce:fembfilter:PDHDFEMBFilter 0.514508 0.519847 0.572532 0.5174 0.0101605 30
produce:wclsdatahd:WireCellToolkit 28.9076 33.4186 36.6704 34.4208 2.613 30
produce:gaushit:GausHitFinder 0.333606 0.519965 0.918911 0.488846 0.117553 30
produce:nhitsfilter:NumberOfHitsFilter 0.00023085 0.000311795 0.000468054 0.000297046 6.35659e-05 30
produce:reco3d:SpacePointSolver 0.133441 0.844589 2.64851 0.637764 0.619744 30
produce:hitpdune:DisambigFromSpacePoints 0.16002 0.377827 0.985329 0.377706 0.188163 30
produce:pandora:StandardPandora 3.81272 9.94799 30.9727 8.79918 5.60947 30
produce:pandoraWriter:StandardPandora 0.0753449 0.126838 0.220394 0.127323 0.0303813 30
produce:pandoraTrack:LArPandoraTrackCreation 0.415197 0.947774 2.37606 0.847345 0.440811 30
produce:pandoraShower:LArPandoraShowerCreation 0.445544 1.09273 3.19439 1.07381 0.531329 30
produce:pandoracalo:Calorimetry 0.240035 0.476813 1.10038 0.441105 0.187084 30
produce:pandoracalonosce:Calorimetry 0.207498 0.432834 0.999041 0.394046 0.173704 30
produce:pandorapid:Chi2ParticleID 0.000929754 0.00143696 0.0027817 0.0013097 0.000364735 30
produce:pandoraShowercalo:ShowerCalorimetry 0.376133 0.999447 3.03411 0.96294 0.493747 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.367773 0.980312 3.09586 0.937409 0.500705 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0709537 0.110341 0.174183 0.111747 0.0205019 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0268785 0.0391806 0.0606863 0.0390605 0.00735538 30
produce:emtrkmichelid:EmTrackMichelId 13.373 21.4961 34.1736 20.8643 4.57335 30
[art]:TriggerResults:TriggerResultInserter 1.3466e-05 1.90462e-05 5.7951e-05 1.63715e-05 8.13751e-06 30
end_path:out1:RootOutput 3.086e-06 4.209e-06 1.7513e-05 3.672e-06 2.50249e-06 30
end_path:michelremoving:michelremoving 0.01462 0.0594244 0.239078 0.0277822 0.0566107 30
end_path:michelremovinggnocchi:michelremoving 0.0142983 0.0570439 0.236708 0.0249739 0.0560005 30
end_path:calibanagnocchi:CalibAnaTree 0.138976 0.357369 0.744231 0.340389 0.132175 30
end_path:out1:RootOutput(write) 0.447318 0.648934 1.03687 0.647686 0.124236 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4936.99 MB
Peak resident set size usage (VmHWM): 2347.04 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0034_dataflow7_datawriter_0_20240906T105948_reco_stage1_reco_stage2_20240926T142747_keepup.root
\tHists: np04hd_raw_run029097_0034_dataflow7_datawriter_0_20240906T105948_reco_stage1_reco_stage2_20240926T142747_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
