Jobsub ID 263764.16@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263764.16@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:50 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce3 |
| Last heartbeat | 2024-09-26 15:05:14 |
| From worker node | Hostname | n196.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:26:11 |
| Input files | hd-protodune:np04hd_raw_run029107_0423_dataflow7_datawriter_0_20240906T225303.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:05:14 |
| Saved logs | justin-logs:263764.16-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
find 17212 neighbours
Iterating with no regularization...
Begin: 8.84383e+06
0 8.21485e+06
1 8.18863e+06
2 8.18716e+06
Now with regularization...
Begin: 7.71766e+06
0 7.71713e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 101759 TrkackID 3 startx, y, z 124.938 600.638 171.456 wireno, peakt tpcno 265 5105.51 2 dist 322.439min T, max_T2826.26 5156.33
no of hits closeby 0 event 101759 TrkackID 16 startx, y, z 105.687 596.391 155.342 wireno, peakt tpcno 99999 -1 -1 dist 99999min T, max_T17.07 2051.6
no of hits closeby 0 event 101759 TrkackID 3 startx, y, z 124.938 600.638 171.456 wireno, peakt tpcno 265 5105.51 2 dist 322.439min T, max_T2826.26 5156.33
no of hits closeby 0 event 101759 TrkackID 16 startx, y, z 105.687 596.391 155.342 wireno, peakt tpcno 99999 -1 -1 dist 99999min T, max_T17.07 2051.6
Begin processing the 29th record. run: 29107 subRun: 1 event: 101767 at 26-Sep-2024 16:01:56 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 59177 traces, 59177 summary
Retagger: tagging trace set: gauss with 42609 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 42609 traces tagged "gauss"
FrameSaver: q=998162 n=746197 tag=gauss
wclsFrameSaver: saving 59177 traces tagged "wiener"
FrameSaver: q=1.09183e+06 n=674685 tag=wiener
750 hits in TPC 0
774 hits in TPC 1
4992 hits in TPC 2
3019 hits in TPC 3
6231 hits in TPC 4
8823 hits in TPC 5
3829 hits in TPC 6
1599 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 60 XUs and 3 XVs -> 0 XUVs
C:0 T:2 39899 XUs and 37194 XVs -> 1097 XUVs
C:0 T:3 46 XUs and 48 XVs -> 0 XUVs
C:0 T:4 470 XUs and 574 XVs -> 19 XUVs
C:0 T:5 55184 XUs and 55962 XVs -> 3812 XUVs
C:0 T:6 22415 XUs and 24339 XVs -> 717 XUVs
C:0 T:7 231 XUs and 247 XVs -> 10 XUVs
5655 XUVs total
1815 collection wire objects
5655 potential space points
Neighbour search...
283575 tests to find 143228 neighbours
Iterating with no regularization...
Begin: 1.71188e+07
0 1.52751e+07
1 1.51274e+07
2 1.51095e+07
3 1.51058e+07
Now with regularization...
Begin: 1.42334e+07
0 1.42302e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 101775 at 26-Sep-2024 16:03:30 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 52463 traces, 52463 summary
Retagger: tagging trace set: gauss with 38898 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 38898 traces tagged "gauss"
FrameSaver: q=832398 n=699751 tag=gauss
wclsFrameSaver: saving 52463 traces tagged "wiener"
FrameSaver: q=896137 n=628849 tag=wiener
2057 hits in TPC 0
1258 hits in TPC 1
3957 hits in TPC 2
1481 hits in TPC 3
3848 hits in TPC 4
5873 hits in TPC 5
6226 hits in TPC 6
2396 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 123 XUs and 305 XVs -> 18 XUVs
C:0 T:2 24255 XUs and 14968 XVs -> 228 XUVs
C:0 T:3 131 XUs and 185 XVs -> 14 XUVs
C:0 T:4 212 XUs and 210 XVs -> 9 XUVs
C:0 T:5 36402 XUs and 20994 XVs -> 993 XUVs
C:0 T:6 17362 XUs and 17978 XVs -> 884 XUVs
C:0 T:7 328 XUs and 223 XVs -> 38 XUVs
2184 XUVs total
1074 collection wire objects
2184 potential space points
Neighbour search...
64660 tests to find 38334 neighbours
Iterating with no regularization...
Begin: 6.33613e+06
0 5.78566e+06
1 5.7225e+06
2 5.71783e+06
Now with regularization...
Begin: 5.32173e+06
0 5.32024e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:04:46 BST Closed output file "np04hd_raw_run029107_0423_dataflow7_datawriter_0_20240906T225303_reco_stage1_reco_stage2_20240926T150446_keepup.root"
26-Sep-2024 16:04:46 BST Closed input file "np04hd_raw_run029107_0423_dataflow7_datawriter_0_20240906T225303_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 70.2917 99.9644 191.213 95.2038 23.922 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000341017 0.00140135 0.00357336 0.000937849 0.000964889 30
produce:fembfilter:PDHDFEMBFilter 0.738106 0.884384 1.28875 0.793641 0.163448 30
produce:wclsdatahd:WireCellToolkit 43.5746 48.5888 59.2804 46.8215 4.53816 30
produce:gaushit:GausHitFinder 0.480399 0.800715 1.23288 0.756703 0.19673 30
produce:nhitsfilter:NumberOfHitsFilter 0.000242368 0.000376261 0.000489226 0.000370271 6.41434e-05 30
produce:reco3d:SpacePointSolver 0.316529 1.09041 2.3472 0.960698 0.526792 30
produce:hitpdune:DisambigFromSpacePoints 0.208664 0.670863 1.72691 0.61368 0.346742 30
produce:pandora:StandardPandora 5.84885 15.6577 71.5082 12.6131 12.2667 30
produce:pandoraWriter:StandardPandora 0.108838 0.16664 0.26527 0.162529 0.034856 30
produce:pandoraTrack:LArPandoraTrackCreation 0.922556 2.19831 9.89576 1.86359 1.61579 30
produce:pandoraShower:LArPandoraShowerCreation 0.803498 2.34064 7.20723 1.961 1.41241 30
produce:pandoracalo:Calorimetry 0.413903 0.910227 3.43872 0.762716 0.549952 30
produce:pandoracalonosce:Calorimetry 0.371542 0.84581 3.17046 0.711684 0.514322 30
produce:pandorapid:Chi2ParticleID 0.00134836 0.00191956 0.00355397 0.00183055 0.000465802 30
produce:pandoraShowercalo:ShowerCalorimetry 0.589973 1.98297 4.99482 1.6913 1.09382 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.59263 1.92994 4.883 1.6283 1.06508 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0867421 0.134772 0.191408 0.129649 0.0278934 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0333673 0.0523452 0.0763915 0.0504637 0.0101789 30
produce:emtrkmichelid:EmTrackMichelId 12.6712 19.5239 28.8404 19.1232 3.98827 30
[art]:TriggerResults:TriggerResultInserter 2.6976e-05 4.03373e-05 7.0089e-05 3.70355e-05 9.54221e-06 30
end_path:out1:RootOutput 6.87e-06 1.0339e-05 2.1568e-05 9.089e-06 3.05374e-06 30
end_path:michelremoving:michelremoving 0.0252836 0.121232 0.350816 0.0645317 0.102613 30
end_path:michelremovinggnocchi:michelremoving 0.02199 0.118474 0.343777 0.0619543 0.100929 30
end_path:calibanagnocchi:CalibAnaTree 0.302568 0.769806 2.30942 0.664557 0.439674 30
end_path:out1:RootOutput(write) 0.568394 0.982707 1.88035 0.877947 0.351227 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5828.26 MB
Peak resident set size usage (VmHWM): 2913.44 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0423_dataflow7_datawriter_0_20240906T225303_reco_stage1_reco_stage2_20240926T150446_keepup.root
\tHists: np04hd_raw_run029107_0423_dataflow7_datawriter_0_20240906T225303_reco_stage1_reco_stage2_20240926T150446_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
