Jobsub ID 263763.114@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263763.114@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:49 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 14:39:45 |
| From worker node | Hostname | wn033.hep |
| cpuinfo | Intel(R) Xeon(R) Gold 6226 CPU @ 2.70GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:50 |
| Input files | hd-protodune:np04hd_raw_run029097_0123_dataflow4_datawriter_0_20240906T125317.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:39:45 |
| Saved logs | justin-logs:263763.114-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
42954 tests to find 24006 neighbours
Iterating with no regularization...
Begin: 5.91823e+06
0 5.68336e+06
1 5.67144e+06
2 5.66944e+06
Now with regularization...
Begin: 5.38763e+06
0 5.38695e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 29740 TrkackID 26 startx, y, z 7.74644 590.854 389.521 wireno, peakt tpcno 380 9.22912 5 dist 0min T, max_T5 353.1
no of hits closeby 2 event 29740 TrkackID 26 startx, y, z 7.74644 590.854 389.521 wireno, peakt tpcno 380 9.22912 5 dist 0min T, max_T5 353.1
Begin processing the 29th record. run: 29097 subRun: 1 event: 29748 at 26-Sep-2024 15:33:00 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67315 traces, 67315 summary
Retagger: tagging trace set: gauss with 51787 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 51787 traces tagged "gauss"
FrameSaver: q=1.14481e+06 n=990545 tag=gauss
wclsFrameSaver: saving 67315 traces tagged "wiener"
FrameSaver: q=1.21224e+06 n=902132 tag=wiener
2869 hits in TPC 0
1922 hits in TPC 1
7927 hits in TPC 2
5659 hits in TPC 3
2832 hits in TPC 4
3681 hits in TPC 5
7867 hits in TPC 6
3598 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 12 XUs and 22 XVs -> 0 XUVs
C:0 T:1 59 XUs and 32 XVs -> 1 XUVs
C:0 T:2 37907 XUs and 41723 XVs -> 2112 XUVs
C:0 T:3 121 XUs and 129 XVs -> 1 XUVs
C:0 T:4 318 XUs and 368 XVs -> 18 XUVs
C:0 T:5 6473 XUs and 7338 XVs -> 307 XUVs
C:0 T:6 22167 XUs and 21827 XVs -> 694 XUVs
C:0 T:7 624 XUs and 621 XVs -> 13 XUVs
3146 XUVs total
1711 collection wire objects
3146 potential space points
Neighbour search...
136486 tests to find 66934 neighbours
Iterating with no regularization...
Begin: 7.57905e+06
0 7.1169e+06
1 7.06546e+06
2 7.05916e+06
Now with regularization...
Begin: 6.58898e+06
0 6.58663e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 29756 at 26-Sep-2024 15:34:22 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78145 traces, 78145 summary
Retagger: tagging trace set: gauss with 58493 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58493 traces tagged "gauss"
FrameSaver: q=1.28955e+06 n=1111670 tag=gauss
wclsFrameSaver: saving 78145 traces tagged "wiener"
FrameSaver: q=1.39168e+06 n=1011495 tag=wiener
3695 hits in TPC 0
3292 hits in TPC 1
6935 hits in TPC 2
4309 hits in TPC 3
2234 hits in TPC 4
5069 hits in TPC 5
9618 hits in TPC 6
6216 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 160 XUs and 99 XVs -> 0 XUVs
C:0 T:2 45732 XUs and 87377 XVs -> 4881 XUVs
C:0 T:3 60 XUs and 65 XVs -> 2 XUVs
C:0 T:4 55 XUs and 60 XVs -> 1 XUVs
C:0 T:5 23619 XUs and 22540 XVs -> 1034 XUVs
C:0 T:6 41508 XUs and 53794 XVs -> 1163 XUVs
C:0 T:7 347 XUs and 449 XVs -> 8 XUVs
7089 XUVs total
1969 collection wire objects
7089 potential space points
Neighbour search...
650563 tests to find 323022 neighbours
Iterating with no regularization...
Begin: 9.32109e+06
0 8.03526e+06
1 7.95176e+06
2 7.93532e+06
3 7.92965e+06
Now with regularization...
Begin: 7.48979e+06
0 7.48565e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 4 event 29756 TrkackID 4 startx, y, z 307.209 542.181 395.176 wireno, peakt tpcno 174 3565.11 5 dist 2.26604min T, max_T5 3901.79
no of hits closeby 4 event 29756 TrkackID 4 startx, y, z 307.209 542.181 395.176 wireno, peakt tpcno 174 3565.11 5 dist 2.26604min T, max_T5 3901.79
26-Sep-2024 15:36:00 BST Closed output file "np04hd_raw_run029097_0123_dataflow4_datawriter_0_20240906T125317_reco_stage1_reco_stage2_20240926T143600_keepup.root"
26-Sep-2024 15:36:00 BST Closed input file "np04hd_raw_run029097_0123_dataflow4_datawriter_0_20240906T125317_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 64.6815 84.7645 106.585 82.8089 11.3352 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000370084 0.00132669 0.0042735 0.00107872 0.000963003 30
produce:fembfilter:PDHDFEMBFilter 0.586463 0.613846 0.839546 0.601419 0.0481039 30
produce:wclsdatahd:WireCellToolkit 43.9402 46.3541 48.9182 46.4213 1.08457 30
produce:gaushit:GausHitFinder 0.377084 0.652491 1.01405 0.68883 0.144918 30
produce:nhitsfilter:NumberOfHitsFilter 0.000257355 0.000393731 0.000568753 0.000394495 7.00749e-05 30
produce:reco3d:SpacePointSolver 0.194562 0.992304 3.46924 0.670415 0.74031 30
produce:hitpdune:DisambigFromSpacePoints 0.206344 0.497237 0.98197 0.484544 0.208568 30
produce:pandora:StandardPandora 4.55496 11.6826 27.9766 10.4846 4.69092 30
produce:pandoraWriter:StandardPandora 0.100241 0.149107 0.201981 0.153678 0.0255973 30
produce:pandoraTrack:LArPandoraTrackCreation 0.676392 2.14967 4.07148 2.13871 0.813592 30
produce:pandoraShower:LArPandoraShowerCreation 0.645004 2.71478 5.26298 2.5132 1.28205 30
produce:pandoracalo:Calorimetry 0.301409 0.872923 1.67953 0.809642 0.331757 30
produce:pandoracalonosce:Calorimetry 0.277311 0.806644 1.43414 0.751067 0.295773 30
produce:pandorapid:Chi2ParticleID 0.0012411 0.00165286 0.00249944 0.00163664 0.000252758 30
produce:pandoraShowercalo:ShowerCalorimetry 0.512672 2.14381 4.36447 2.04661 0.994775 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.505451 2.06042 3.95578 1.90763 0.958394 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0595672 0.110015 0.168304 0.108139 0.0276519 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0260688 0.0444064 0.0653616 0.043031 0.0103405 30
produce:emtrkmichelid:EmTrackMichelId 7.58222 11.2711 14.9159 11.066 1.96267 30
[art]:TriggerResults:TriggerResultInserter 2.1213e-05 4.03661e-05 6.9832e-05 3.99225e-05 9.72594e-06 30
end_path:out1:RootOutput 7.323e-06 1.86804e-05 0.000211012 1.16175e-05 3.58338e-05 30
end_path:michelremoving:michelremoving 0.0214454 0.097542 0.335754 0.0530131 0.0846475 30
end_path:michelremovinggnocchi:michelremoving 0.0171963 0.0865566 0.33469 0.0509406 0.0781179 30
end_path:calibanagnocchi:CalibAnaTree 0.160737 0.595442 1.09823 0.585829 0.210886 30
end_path:out1:RootOutput(write) 0.381424 0.663263 0.976431 0.664275 0.130261 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5776.42 MB
Peak resident set size usage (VmHWM): 2603.09 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0123_dataflow4_datawriter_0_20240906T125317_reco_stage1_reco_stage2_20240926T143600_keepup.root
\tHists: np04hd_raw_run029097_0123_dataflow4_datawriter_0_20240906T125317_reco_stage1_reco_stage2_20240926T143600_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
