Jobsub ID 263685.105@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263685.105@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:08:36 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:37:07 |
| From worker node | Hostname | n255.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:09:51 |
| Input files | hd-protodune:np04hd_raw_run029097_0266_dataflow2_datawriter_0_20240906T155126.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:37:07 |
| Saved logs | justin-logs:263685.105-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
TPC 3
2505 hits in TPC 4
4042 hits in TPC 5
3041 hits in TPC 6
2483 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 757 XUs and 621 XVs -> 6 XUVs
C:0 T:2 12894 XUs and 18671 XVs -> 486 XUVs
C:0 T:3 73 XUs and 125 XVs -> 4 XUVs
C:0 T:4 194 XUs and 179 XVs -> 11 XUVs
C:0 T:5 6384 XUs and 9951 XVs -> 206 XUVs
C:0 T:6 10450 XUs and 8133 XVs -> 198 XUVs
C:0 T:7 32 XUs and 64 XVs -> 4 XUVs
915 XUVs total
625 collection wire objects
915 potential space points
Neighbour search...
9189 tests to find 5442 neighbours
Iterating with no regularization...
Begin: 3.46045e+06
0 3.31465e+06
1 3.30989e+06
2 3.30955e+06
Now with regularization...
Begin: 3.12942e+06
0 3.12932e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 64066 at 26-Sep-2024 15:27:14 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 52758 traces, 52758 summary
Retagger: tagging trace set: gauss with 41612 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41612 traces tagged "gauss"
FrameSaver: q=912418 n=832596 tag=gauss
wclsFrameSaver: saving 52758 traces tagged "wiener"
FrameSaver: q=964147 n=759943 tag=wiener
2680 hits in TPC 0
2254 hits in TPC 1
4074 hits in TPC 2
3041 hits in TPC 3
3264 hits in TPC 4
5156 hits in TPC 5
4763 hits in TPC 6
4332 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 69 XUs and 64 XVs -> 0 XUVs
C:0 T:2 10210 XUs and 8181 XVs -> 376 XUVs
C:0 T:3 8 XUs and 11 XVs -> 0 XUVs
C:0 T:4 91 XUs and 103 XVs -> 5 XUVs
C:0 T:5 18186 XUs and 23390 XVs -> 604 XUVs
C:0 T:6 16954 XUs and 10561 XVs -> 453 XUVs
C:0 T:7 767 XUs and 549 XVs -> 22 XUVs
1460 XUVs total
956 collection wire objects
1460 potential space points
Neighbour search...
24722 tests to find 13134 neighbours
Iterating with no regularization...
Begin: 5.41105e+06
0 5.0956e+06
1 5.08153e+06
2 5.08036e+06
Now with regularization...
Begin: 4.80738e+06
0 4.80693e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 64082 at 26-Sep-2024 15:28:15 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 51687 traces, 51687 summary
Retagger: tagging trace set: gauss with 40934 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40934 traces tagged "gauss"
FrameSaver: q=880505 n=834254 tag=gauss
wclsFrameSaver: saving 51687 traces tagged "wiener"
FrameSaver: q=942007 n=774790 tag=wiener
4065 hits in TPC 0
5069 hits in TPC 1
5089 hits in TPC 2
3695 hits in TPC 3
3545 hits in TPC 4
3543 hits in TPC 5
3168 hits in TPC 6
2287 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 527 XUs and 457 XVs -> 75 XUVs
C:0 T:2 14902 XUs and 10898 XVs -> 476 XUVs
C:0 T:3 160 XUs and 343 XVs -> 3 XUVs
C:0 T:4 200 XUs and 210 XVs -> 7 XUVs
C:0 T:5 7798 XUs and 6003 XVs -> 292 XUVs
C:0 T:6 4577 XUs and 4961 XVs -> 262 XUVs
C:0 T:7 131 XUs and 141 XVs -> 15 XUVs
1130 XUVs total
893 collection wire objects
1130 potential space points
Neighbour search...
10926 tests to find 6172 neighbours
Iterating with no regularization...
Begin: 3.28317e+06
0 3.18904e+06
1 3.18862e+06
Now with regularization...
Begin: 2.93898e+06
0 2.93893e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:29:10 BST Closed output file "np04hd_raw_run029097_0266_dataflow2_datawriter_0_20240906T155126_reco_stage1_reco_stage2_20240926T142910_keepup.root"
26-Sep-2024 15:29:10 BST Closed input file "np04hd_raw_run029097_0266_dataflow2_datawriter_0_20240906T155126_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 40.1412 58.1931 106.702 56.4862 12.9051 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000390831 0.00183266 0.00372576 0.00162587 0.000836748 30
produce:fembfilter:PDHDFEMBFilter 0.445461 0.477405 0.673066 0.457218 0.0624795 30
produce:wclsdatahd:WireCellToolkit 24.5801 26.3376 36.2345 25.5108 2.69784 30
produce:gaushit:GausHitFinder 0.212737 0.444782 1.0584 0.427982 0.15453 30
produce:nhitsfilter:NumberOfHitsFilter 0.000174581 0.000251251 0.00036422 0.000247289 4.52717e-05 30
produce:reco3d:SpacePointSolver 0.111172 0.482578 1.70653 0.364086 0.3941 30
produce:hitpdune:DisambigFromSpacePoints 0.0753277 0.29297 1.04164 0.242305 0.194703 30
produce:pandora:StandardPandora 2.60744 7.47762 20.0602 6.48736 3.79758 30
produce:pandoraWriter:StandardPandora 0.0427673 0.103497 0.260379 0.0939831 0.0396785 30
produce:pandoraTrack:LArPandoraTrackCreation 0.156891 0.784047 2.4806 0.714002 0.416863 30
produce:pandoraShower:LArPandoraShowerCreation 0.133078 0.938746 3.77142 0.839942 0.671367 30
produce:pandoracalo:Calorimetry 0.107005 0.407776 1.2892 0.381553 0.203409 30
produce:pandoracalonosce:Calorimetry 0.0878646 0.36692 1.01479 0.344808 0.167165 30
produce:pandorapid:Chi2ParticleID 0.000850473 0.00127829 0.0025374 0.00123499 0.000334288 30
produce:pandoraShowercalo:ShowerCalorimetry 0.128584 0.871507 3.79179 0.776992 0.651121 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.122217 0.829383 3.21932 0.751915 0.573451 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.036401 0.084612 0.255123 0.0812438 0.0368277 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0109505 0.0295215 0.0853368 0.0272226 0.0125371 30
produce:emtrkmichelid:EmTrackMichelId 9.75441 17.2281 28.2879 17.2092 4.30424 30
[art]:TriggerResults:TriggerResultInserter 1.562e-05 2.58971e-05 5.5004e-05 2.3184e-05 8.90084e-06 30
end_path:out1:RootOutput 6.743e-06 9.36003e-06 1.8394e-05 9.092e-06 2.24004e-06 30
end_path:michelremoving:michelremoving 0.00904304 0.0495583 0.382188 0.0195213 0.0760878 30
end_path:michelremovinggnocchi:michelremoving 0.0073146 0.0489426 0.397366 0.0173174 0.0785108 30
end_path:calibanagnocchi:CalibAnaTree 0.0592661 0.298914 0.685691 0.27903 0.138963 30
end_path:out1:RootOutput(write) 0.261319 0.529161 1.05274 0.517541 0.159127 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4912.5 MB
Peak resident set size usage (VmHWM): 2579.72 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0266_dataflow2_datawriter_0_20240906T155126_reco_stage1_reco_stage2_20240926T142910_keepup.root
\tHists: np04hd_raw_run029097_0266_dataflow2_datawriter_0_20240906T155126_reco_stage1_reco_stage2_20240926T142910_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
