Jobsub ID 78544.0@justin-prod-sched02.dune.hep.ac.uk
Jobsub ID | 78544.0@justin-prod-sched02.dune.hep.ac.uk |
Workflow ID | 3451 |
Stage ID | 1 |
User name | calcuttj@fnal.gov |
HTCondor Group | group_dune.prod_mcsim |
Requested | Processors | 1 |
RSS bytes | 4193255424 (3999 MiB) |
Wall seconds limit | 18000 (5 hours) |
Submitted time | 2024-09-26 10:24:11 |
Site | UK_Sheffield |
Entry | DUNE_UK_Sheffield_lcgce1 |
Last heartbeat | 2024-09-26 12:49:58 |
From worker node | Hostname | wn033.hep |
cpuinfo | Intel(R) Xeon(R) Gold 6226 CPU @ 2.70GHz |
OS release | Scientific Linux release 7.9 (Nitrogen) |
Processors | 1 |
RSS bytes | 4193255424 (3999 MiB) |
Wall seconds limit | 257400 (71 hours) |
Inner Apptainer? | True |
Job state | outputting_failed |
Allocator name | justin-allocator-pro.dune.hep.ac.uk |
Started | 2024-09-26 11:08:13 |
Input files | hd-protodune:np04hd_raw_run029107_2162_dataflow3_datawriter_0_20240908T023939.hdf5
|
Jobscript | Exit code | 1 |
Real time | 0m (0s) |
CPU time | 0m (0s = 0%) |
Outputting started | |
Output files | |
Finished | 2024-09-26 12:49:58 |
List job events Wrapper job log |
Jobscript log (last 10,000 characters)
7.16706e+06
3 7.16304e+06
Now with regularization...
Begin: 6.68322e+06
0 6.68002e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 519131 at 26-Sep-2024 13:15:15 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67976 traces, 67976 summary
Retagger: tagging trace set: gauss with 50344 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50344 traces tagged "gauss"
FrameSaver: q=1.16004e+06 n=942223 tag=gauss
wclsFrameSaver: saving 67976 traces tagged "wiener"
FrameSaver: q=1.26762e+06 n=858439 tag=wiener
6783 hits in TPC 0
3863 hits in TPC 1
6872 hits in TPC 2
3867 hits in TPC 3
4032 hits in TPC 4
4365 hits in TPC 5
6667 hits in TPC 6
1957 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 6813 XUs and 4793 XVs -> 365 XUVs
C:0 T:2 50944 XUs and 64030 XVs -> 4600 XUVs
C:0 T:3 313 XUs and 395 XVs -> 22 XUVs
C:0 T:4 188 XUs and 242 XVs -> 19 XUVs
C:0 T:5 13186 XUs and 17370 XVs -> 519 XUVs
C:0 T:6 30874 XUs and 49444 XVs -> 4391 XUVs
C:0 T:7 61 XUs and 67 XVs -> 2 XUVs
9918 XUVs total
1823 collection wire objects
9918 potential space points
Neighbour search...
2022424 tests to find 865236 neighbours
Iterating with no regularization...
Begin: 1.50095e+07
0 1.39037e+07
1 1.38309e+07
2 1.38239e+07
Now with regularization...
Begin: 1.33941e+07
0 1.33908e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 519139 at 26-Sep-2024 13:16:48 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66596 traces, 66596 summary
Retagger: tagging trace set: gauss with 50166 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50166 traces tagged "gauss"
FrameSaver: q=1.20009e+06 n=996732 tag=gauss
wclsFrameSaver: saving 66596 traces tagged "wiener"
FrameSaver: q=1.27432e+06 n=909065 tag=wiener
3666 hits in TPC 0
4100 hits in TPC 1
10536 hits in TPC 2
4855 hits in TPC 3
1563 hits in TPC 4
4408 hits in TPC 5
5329 hits in TPC 6
2949 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 295 XUs and 202 XVs -> 22 XUVs
C:0 T:2 28394 XUs and 34314 XVs -> 1430 XUVs
C:0 T:3 173 XUs and 238 XVs -> 12 XUVs
C:0 T:4 0 XUs and 0 XVs -> 0 XUVs
C:0 T:5 27022 XUs and 32198 XVs -> 1949 XUVs
C:0 T:6 9532 XUs and 9913 XVs -> 622 XUVs
C:0 T:7 20 XUs and 24 XVs -> 0 XUVs
4035 XUVs total
1574 collection wire objects
4035 potential space points
Neighbour search...
341225 tests to find 177756 neighbours
Iterating with no regularization...
Begin: 3.90759e+07
0 3.76034e+07
1 3.7461e+07
2 3.74536e+07
Now with regularization...
Begin: 3.56365e+07
0 3.56308e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 519139 TrkackID 0 startx, y, z 236.118 606.434 118.421 wireno, peakt tpcno 306 3006.89 2 dist 283.07min T, max_T24.5268 3047.07
no of hits closeby 5 event 519139 TrkackID 4 startx, y, z -23.4368 596.72 228.443 wireno, peakt tpcno 72 3956.6 6 dist 1.63773min T, max_T3900.63 5050.35
no of hits closeby 0 event 519139 TrkackID 0 startx, y, z 236.118 606.434 118.421 wireno, peakt tpcno 306 3006.89 2 dist 283.07min T, max_T24.5268 3047.07
no of hits closeby 5 event 519139 TrkackID 4 startx, y, z -23.4368 596.72 228.443 wireno, peakt tpcno 72 3956.6 6 dist 1.63773min T, max_T3900.63 5050.35
26-Sep-2024 13:18:18 BST Closed output file "np04hd_raw_run029107_2162_dataflow3_datawriter_0_20240908T023939_reco_stage1_reco_stage2_20240926T121818_keepup.root"
26-Sep-2024 13:18:18 BST Closed input file "np04hd_raw_run029107_2162_dataflow3_datawriter_0_20240908T023939_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 64.3612 81.3631 111.412 79.4915 11.1164 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000361778 0.00135659 0.00394891 0.000908748 0.00108333 30
produce:fembfilter:PDHDFEMBFilter 0.598911 0.621745 0.739883 0.611985 0.0286231 30
produce:wclsdatahd:WireCellToolkit 42.8748 46.2356 50.6111 46.1703 1.82924 30
produce:gaushit:GausHitFinder 0.335944 0.606554 0.874398 0.610575 0.117342 30
produce:nhitsfilter:NumberOfHitsFilter 0.000250026 0.000391334 0.000577003 0.000396164 6.47242e-05 30
produce:reco3d:SpacePointSolver 0.192411 0.857111 2.34421 0.712685 0.496212 30
produce:hitpdune:DisambigFromSpacePoints 0.146197 0.426447 0.892984 0.408296 0.156653 30
produce:pandora:StandardPandora 5.00993 10.2611 18.0971 9.31791 3.24582 30
produce:pandoraWriter:StandardPandora 0.0981184 0.149394 0.229452 0.146043 0.0291297 30
produce:pandoraTrack:LArPandoraTrackCreation 0.707232 2.05519 4.90754 1.85585 0.952604 30
produce:pandoraShower:LArPandoraShowerCreation 0.471865 2.32077 6.43786 2.13368 1.27999 30
produce:pandoracalo:Calorimetry 0.30336 0.817878 1.80824 0.747708 0.366305 30
produce:pandoracalonosce:Calorimetry 0.301872 0.762556 1.638 0.691742 0.336998 30
produce:pandorapid:Chi2ParticleID 0.00120127 0.00166945 0.00378353 0.00150619 0.000482176 30
produce:pandoraShowercalo:ShowerCalorimetry 0.417134 1.86714 4.79338 1.62805 0.987804 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.417923 1.78737 4.54313 1.56574 0.884687 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0577911 0.107256 0.152102 0.106488 0.0221666 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0239966 0.0438973 0.0652163 0.0433664 0.00950652 30
produce:emtrkmichelid:EmTrackMichelId 6.83045 10.9542 16.6801 10.5363 2.31918 30
[art]:TriggerResults:TriggerResultInserter 3.2525e-05 4.69599e-05 9.4599e-05 4.4276e-05 1.14086e-05 30
end_path:out1:RootOutput 9.518e-06 1.28116e-05 2.9451e-05 1.2174e-05 3.64183e-06 30
end_path:michelremoving:michelremoving 0.0233711 0.0605294 0.22509 0.0401334 0.0500903 30
end_path:michelremovinggnocchi:michelremoving 0.0215786 0.0577107 0.217925 0.0362659 0.0476291 30
end_path:calibanagnocchi:CalibAnaTree 0.254247 0.529999 1.16655 0.463942 0.210689 30
end_path:out1:RootOutput(write) 0.391978 0.612296 0.877998 0.600828 0.123935 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5879.56 MB
Peak resident set size usage (VmHWM): 2699.73 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2162_dataflow3_datawriter_0_20240908T023939_reco_stage1_reco_stage2_20240926T121818_keepup.root
\tHists: np04hd_raw_run029107_2162_dataflow3_datawriter_0_20240908T023939_reco_stage1_reco_stage2_20240926T121818_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata