Jobsub ID 263763.78@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263763.78@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:49 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 14:52:50 |
| From worker node | Hostname | wn093.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:26:07 |
| Input files | hd-protodune:np04hd_raw_run029097_0128_dataflow1_datawriter_0_20240906T130052.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:52:50 |
| Saved logs | justin-logs:263763.78-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
C:0 T:4 1 XUs and 1 XVs -> 0 XUVs
C:0 T:5 7984 XUs and 9446 XVs -> 234 XUVs
C:0 T:6 12953 XUs and 10715 XVs -> 418 XUVs
C:0 T:7 93 XUs and 32 XVs -> 0 XUVs
1897 XUVs total
1032 collection wire objects
1897 potential space points
Neighbour search...
47495 tests to find 24446 neighbours
Iterating with no regularization...
Begin: 6.92376e+06
0 6.35898e+06
1 6.32185e+06
2 6.31764e+06
Now with regularization...
Begin: 5.99662e+06
0 5.99473e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 8 event 30937 TrkackID 7 startx, y, z 58.4063 595.243 402.189 wireno, peakt tpcno 456 3503.08 5 dist 27.0584min T, max_T3312.11 5479.79
no of hits closeby 8 event 30937 TrkackID 7 startx, y, z 58.4063 595.243 402.189 wireno, peakt tpcno 456 3503.08 5 dist 27.0584min T, max_T3312.11 5479.79
Begin processing the 29th record. run: 29097 subRun: 1 event: 30945 at 26-Sep-2024 15:49:21 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 62521 traces, 62521 summary
Retagger: tagging trace set: gauss with 47966 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47966 traces tagged "gauss"
FrameSaver: q=1.06769e+06 n=933508 tag=gauss
wclsFrameSaver: saving 62521 traces tagged "wiener"
FrameSaver: q=1.125e+06 n=848162 tag=wiener
3358 hits in TPC 0
3087 hits in TPC 1
7311 hits in TPC 2
5666 hits in TPC 3
3383 hits in TPC 4
6346 hits in TPC 5
2449 hits in TPC 6
1588 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 251 XUs and 245 XVs -> 14 XUVs
C:0 T:2 24881 XUs and 21837 XVs -> 845 XUVs
C:0 T:3 241 XUs and 314 XVs -> 8 XUVs
C:0 T:4 133 XUs and 148 XVs -> 11 XUVs
C:0 T:5 23038 XUs and 16033 XVs -> 576 XUVs
C:0 T:6 3979 XUs and 4544 XVs -> 146 XUVs
C:0 T:7 46 XUs and 106 XVs -> 3 XUVs
1603 XUVs total
999 collection wire objects
1603 potential space points
Neighbour search...
22641 tests to find 13978 neighbours
Iterating with no regularization...
Begin: 1.08833e+07
0 1.02523e+07
1 1.02286e+07
2 1.02248e+07
Now with regularization...
Begin: 9.6441e+06
0 9.64283e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 30945 TrkackID 0 startx, y, z 215.932 402.835 461.022 wireno, peakt tpcno 53 2.49859 5 dist 0min T, max_T2.49859 2893.73
no of hits closeby 0 event 30945 TrkackID 0 startx, y, z 215.932 402.835 461.022 wireno, peakt tpcno 53 2.49859 5 dist 0min T, max_T2.49859 2893.73
Begin processing the 30th record. run: 29097 subRun: 1 event: 30953 at 26-Sep-2024 15:50:57 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 51046 traces, 51046 summary
Retagger: tagging trace set: gauss with 40932 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40932 traces tagged "gauss"
FrameSaver: q=942823 n=827898 tag=gauss
wclsFrameSaver: saving 51046 traces tagged "wiener"
FrameSaver: q=989104 n=760275 tag=wiener
2705 hits in TPC 0
2202 hits in TPC 1
4485 hits in TPC 2
3495 hits in TPC 3
3722 hits in TPC 4
6894 hits in TPC 5
2653 hits in TPC 6
2800 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2346 XUs and 1072 XVs -> 46 XUVs
C:0 T:2 16501 XUs and 11856 XVs -> 339 XUVs
C:0 T:3 71 XUs and 71 XVs -> 2 XUVs
C:0 T:4 392 XUs and 726 XVs -> 117 XUVs
C:0 T:5 16811 XUs and 19346 XVs -> 549 XUVs
C:0 T:6 4927 XUs and 5042 XVs -> 272 XUVs
C:0 T:7 22 XUs and 18 XVs -> 0 XUVs
1325 XUVs total
938 collection wire objects
1325 potential space points
Neighbour search...
20723 tests to find 11656 neighbours
Iterating with no regularization...
Begin: 8.1361e+06
0 7.78365e+06
1 7.77053e+06
2 7.7697e+06
Now with regularization...
Begin: 7.27338e+06
0 7.27311e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:52:27 BST Closed output file "np04hd_raw_run029097_0128_dataflow1_datawriter_0_20240906T130052_reco_stage1_reco_stage2_20240926T145227_keepup.root"
26-Sep-2024 15:52:27 BST Closed input file "np04hd_raw_run029097_0128_dataflow1_datawriter_0_20240906T130052_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 78.8448 109.599 187.897 102.28 23.4682 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000623965 0.00280192 0.00483135 0.00303078 0.00130864 30
produce:fembfilter:PDHDFEMBFilter 0.671211 0.717271 0.831661 0.707936 0.0301464 30
produce:wclsdatahd:WireCellToolkit 45.7967 51.7617 60.565 48.6714 5.46696 30
produce:gaushit:GausHitFinder 0.411265 0.797179 1.39793 0.777876 0.191017 30
produce:nhitsfilter:NumberOfHitsFilter 0.000296554 0.000419389 0.00064503 0.00040129 8.89701e-05 30
produce:reco3d:SpacePointSolver 0.178721 1.00686 4.11119 0.850954 0.771811 30
produce:hitpdune:DisambigFromSpacePoints 0.13761 0.554871 1.29715 0.506614 0.250206 30
produce:pandora:StandardPandora 5.7602 13.8655 36.1713 12.2375 6.06656 30
produce:pandoraWriter:StandardPandora 0.100243 0.169663 0.296382 0.159128 0.0407961 30
produce:pandoraTrack:LArPandoraTrackCreation 0.638035 1.73614 5.15042 1.49879 0.946039 30
produce:pandoraShower:LArPandoraShowerCreation 0.511664 2.1663 8.13383 1.54666 1.59825 30
produce:pandoracalo:Calorimetry 0.321636 0.822259 1.98381 0.729749 0.35407 30
produce:pandoracalonosce:Calorimetry 0.293611 0.735899 1.85721 0.648042 0.320867 30
produce:pandorapid:Chi2ParticleID 0.00154869 0.00209891 0.00403646 0.00196297 0.000525279 30
produce:pandoraShowercalo:ShowerCalorimetry 0.455864 1.81071 5.18075 1.37835 1.04231 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.445786 1.7221 5.01294 1.32268 0.990744 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0901444 0.152846 0.45162 0.138569 0.0638523 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0343283 0.0531807 0.0926616 0.0512736 0.01187 30
produce:emtrkmichelid:EmTrackMichelId 18.8019 29.8125 54.6501 26.4321 7.87057 30
[art]:TriggerResults:TriggerResultInserter 2.3675e-05 5.38185e-05 0.000149973 5.35175e-05 2.40542e-05 30
end_path:out1:RootOutput 9.184e-06 1.53059e-05 4.5486e-05 1.21465e-05 7.3429e-06 30
end_path:michelremoving:michelremoving 0.0279711 0.0850394 0.396543 0.0510278 0.0774333 30
end_path:michelremovinggnocchi:michelremoving 0.0254742 0.0775981 0.377542 0.0401836 0.0752006 30
end_path:calibanagnocchi:CalibAnaTree 0.317552 0.598132 1.37276 0.511472 0.241904 30
end_path:out1:RootOutput(write) 0.517743 0.810372 1.3501 0.800079 0.173156 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4921.09 MB
Peak resident set size usage (VmHWM): 2341.13 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0128_dataflow1_datawriter_0_20240906T130052_reco_stage1_reco_stage2_20240926T145227_keepup.root
\tHists: np04hd_raw_run029097_0128_dataflow1_datawriter_0_20240906T130052_reco_stage1_reco_stage2_20240926T145227_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
