Jobsub ID 263855.1@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263855.1@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3458 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:52:08 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 15:39:38 |
| From worker node | Hostname | wn098.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2698 v4 @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:54:06 |
| Input files | hd-protodune:np04hd_raw_run029296_0085_dataflow1_datawriter_0_20240926T041040.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:39:38 |
| Saved logs | justin-logs:263855.1-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
terating with no regularization...
Begin: 5.37257e+06
0 4.8912e+06
1 4.86545e+06
2 4.86003e+06
3 4.85806e+06
Now with regularization...
Begin: 4.60108e+06
0 4.59984e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 6703 TrkackID 2 startx, y, z 127.895 408.825 3.70991 wireno, peakt tpcno 183 3300.15 2 dist 349.112min T, max_T11 3342.5
no of hits closeby 0 event 6703 TrkackID 2 startx, y, z 127.895 408.825 3.70991 wireno, peakt tpcno 183 3300.15 2 dist 349.112min T, max_T11 3342.5
Begin processing the 38th record. run: 29296 subRun: 1 event: 6705 at 26-Sep-2024 16:35:11 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 85857 traces, 85857 summary
Retagger: tagging trace set: gauss with 60627 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 60627 traces tagged "gauss"
FrameSaver: q=1.349e+06 n=1047469 tag=gauss
wclsFrameSaver: saving 85857 traces tagged "wiener"
FrameSaver: q=1.4643e+06 n=926543 tag=wiener
2163 hits in TPC 0
2210 hits in TPC 1
11426 hits in TPC 2
7310 hits in TPC 3
2897 hits in TPC 4
4835 hits in TPC 5
7657 hits in TPC 6
4377 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 174 XUs and 110 XVs -> 41 XUVs
C:0 T:2 91279 XUs and 72716 XVs -> 1964 XUVs
C:0 T:3 757 XUs and 730 XVs -> 15 XUVs
C:0 T:4 67 XUs and 137 XVs -> 3 XUVs
C:0 T:5 19445 XUs and 28403 XVs -> 1245 XUVs
C:0 T:6 50251 XUs and 23310 XVs -> 587 XUVs
C:0 T:7 610 XUs and 482 XVs -> 12 XUVs
3867 XUVs total
1795 collection wire objects
3867 potential space points
Neighbour search...
200325 tests to find 90192 neighbours
Iterating with no regularization...
Begin: 2.74251e+07
0 2.40669e+07
1 2.37106e+07
2 2.36915e+07
Now with regularization...
Begin: 2.26488e+07
0 2.26393e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 39th record. run: 29296 subRun: 1 event: 6707 at 26-Sep-2024 16:37:35 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 52070 traces, 52070 summary
Retagger: tagging trace set: gauss with 38801 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 38801 traces tagged "gauss"
FrameSaver: q=853926 n=739914 tag=gauss
wclsFrameSaver: saving 52070 traces tagged "wiener"
FrameSaver: q=922078 n=679983 tag=wiener
3555 hits in TPC 0
6171 hits in TPC 1
4336 hits in TPC 2
1899 hits in TPC 3
2796 hits in TPC 4
3970 hits in TPC 5
2433 hits in TPC 6
1766 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1920 XUs and 2351 XVs -> 50 XUVs
C:0 T:2 13180 XUs and 10820 XVs -> 512 XUVs
C:0 T:3 62 XUs and 78 XVs -> 1 XUVs
C:0 T:4 196 XUs and 191 XVs -> 7 XUVs
C:0 T:5 10779 XUs and 5878 XVs -> 143 XUVs
C:0 T:6 2142 XUs and 4337 XVs -> 126 XUVs
C:0 T:7 28 XUs and 28 XVs -> 0 XUVs
839 XUVs total
465 collection wire objects
839 potential space points
Neighbour search...
19455 tests to find 8074 neighbours
Iterating with no regularization...
Begin: 4.11428e+06
0 3.74493e+06
1 3.7259e+06
2 3.72431e+06
Now with regularization...
Begin: 3.52593e+06
0 3.52544e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 6707 TrkackID 2 startx, y, z 242.601 605.697 375.512 wireno, peakt tpcno 0 4265.05 5 dist 0.78668min T, max_T2213.9 5301.8
no of hits closeby 0 event 6707 TrkackID 2 startx, y, z 242.601 605.697 375.512 wireno, peakt tpcno 0 4265.05 5 dist 0.78668min T, max_T2213.9 5301.8
26-Sep-2024 16:39:17 BST Closed output file "np04hd_raw_run029296_0085_dataflow1_datawriter_0_20240926T041040_reco_stage1_reco_stage2_20240926T153917_keepup.root"
26-Sep-2024 16:39:17 BST Closed input file "np04hd_raw_run029296_0085_dataflow1_datawriter_0_20240926T041040_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 91.4 120.463 181.63 116.639 22.6383 39
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000512205 0.00284876 0.00476027 0.00322083 0.00117748 39
produce:fembfilter:PDHDFEMBFilter 0.719765 0.745649 0.925979 0.741268 0.0305128 39
produce:wclsdatahd:WireCellToolkit 52.4914 56.0407 58.4986 56.259 1.09907 39
produce:gaushit:GausHitFinder 0.432502 0.817965 1.28681 0.797651 0.184774 39
produce:nhitsfilter:NumberOfHitsFilter 0.0002228 0.000392045 0.00071629 0.000381093 9.57575e-05 39
produce:reco3d:SpacePointSolver 0.162706 1.15246 6.27857 0.575974 1.55664 39
produce:hitpdune:DisambigFromSpacePoints 0.150374 0.64044 2.14796 0.567437 0.399334 39
produce:pandora:StandardPandora 6.32885 16.0694 39.814 13.9943 8.13207 39
produce:pandoraWriter:StandardPandora 0.0978652 0.163856 0.280087 0.157356 0.043631 39
produce:pandoraTrack:LArPandoraTrackCreation 0.636866 1.66111 4.33136 1.53353 0.739788 39
produce:pandoraShower:LArPandoraShowerCreation 0.583215 1.9608 4.92206 1.62691 1.02668 39
produce:pandoracalo:Calorimetry 0.333333 0.836271 2.2212 0.775414 0.352174 39
produce:pandoracalonosce:Calorimetry 0.300941 0.761214 2.03035 0.717042 0.324813 39
produce:pandorapid:Chi2ParticleID 0.00126055 0.00191616 0.00330441 0.00190863 0.000398816 39
produce:pandoraShowercalo:ShowerCalorimetry 0.553082 1.87772 5.47173 1.6776 0.992745 39
produce:pandoraShowercalonosce:ShowerCalorimetry 0.537886 1.81521 5.05801 1.64315 0.926043 39
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0826017 0.130347 0.223017 0.128221 0.0312185 39
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0313796 0.0494008 0.0809792 0.0481161 0.0114231 39
produce:emtrkmichelid:EmTrackMichelId 22.5122 34.054 54.6231 33.8394 8.00815 39
[art]:TriggerResults:TriggerResultInserter 2.252e-05 3.44806e-05 7.1692e-05 3.2672e-05 9.3252e-06 39
end_path:out1:RootOutput 6.565e-06 1.46592e-05 0.000209495 8.88e-06 3.18266e-05 39
end_path:michelremoving:michelremoving 0.0231991 0.0785186 0.193055 0.0513226 0.0541347 39
end_path:michelremovinggnocchi:michelremoving 0.0208303 0.0724561 0.191312 0.0445007 0.0514527 39
end_path:calibanagnocchi:CalibAnaTree 0.217445 0.621815 1.62737 0.586687 0.263927 39
end_path:out1:RootOutput(write) 0.477776 0.799325 1.33902 0.788325 0.201207 39
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4939.86 MB
Peak resident set size usage (VmHWM): 2308.29 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029296_0085_dataflow1_datawriter_0_20240926T041040_reco_stage1_reco_stage2_20240926T153917_keepup.root
\tHists: np04hd_raw_run029296_0085_dataflow1_datawriter_0_20240926T041040_reco_stage1_reco_stage2_20240926T153917_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
