Jobsub ID 263865.20@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263865.20@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:56:43 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 15:34:48 |
| From worker node | Hostname | wn024.hep |
| cpuinfo | Intel(R) Xeon(R) Gold 5115 CPU @ 2.40GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:57:54 |
| Input files | hd-protodune:np04hd_raw_run029097_0121_dataflow5_datawriter_0_20240906T124944.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:34:48 |
| Saved logs | justin-logs:263865.20-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
--------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 7 event 29261 TrkackID 15 startx, y, z 7.93379 204.886 447.203 wireno, peakt tpcno 409 1138 5 dist 0.937876min T, max_T1057.64 1836.92
no of hits closeby 7 event 29261 TrkackID 15 startx, y, z 7.93379 204.886 447.203 wireno, peakt tpcno 409 1138 5 dist 0.937876min T, max_T1057.64 1836.92
Begin processing the 29th record. run: 29097 subRun: 1 event: 29269 at 26-Sep-2024 16:29:43 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 61986 traces, 61986 summary
Retagger: tagging trace set: gauss with 45672 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 45672 traces tagged "gauss"
FrameSaver: q=933221 n=837385 tag=gauss
wclsFrameSaver: saving 61986 traces tagged "wiener"
FrameSaver: q=1.00183e+06 n=746798 tag=wiener
3989 hits in TPC 0
3811 hits in TPC 1
7179 hits in TPC 2
3213 hits in TPC 3
2412 hits in TPC 4
4764 hits in TPC 5
5779 hits in TPC 6
1773 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3205 XUs and 702 XVs -> 109 XUVs
C:0 T:2 40374 XUs and 50123 XVs -> 2010 XUVs
C:0 T:3 39 XUs and 57 XVs -> 3 XUVs
C:0 T:4 28 XUs and 35 XVs -> 4 XUVs
C:0 T:5 17116 XUs and 8171 XVs -> 369 XUVs
C:0 T:6 40492 XUs and 48680 XVs -> 975 XUVs
C:0 T:7 254 XUs and 286 XVs -> 9 XUVs
3479 XUVs total
1200 collection wire objects
3479 potential space points
Neighbour search...
220395 tests to find 115540 neighbours
Iterating with no regularization...
Begin: 6.14515e+06
0 5.86755e+06
1 5.84721e+06
2 5.84409e+06
Now with regularization...
Begin: 5.6544e+06
0 5.65313e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 29269 TrkackID 0 startx, y, z 47.0615 514.716 7.7173 wireno, peakt tpcno 450 1838.15 5 dist 0min T, max_T1549.97 2195.04
no of hits closeby 0 event 29269 TrkackID 15 startx, y, z 25.3616 331.933 24.9159 wireno, peakt tpcno 91 299.98 2 dist 90.2176min T, max_T3.44447 325.461
no of hits closeby 0 event 29269 TrkackID 0 startx, y, z 47.0615 514.716 7.7173 wireno, peakt tpcno 450 1838.15 5 dist 0min T, max_T1549.97 2195.04
no of hits closeby 0 event 29269 TrkackID 15 startx, y, z 25.3616 331.933 24.9159 wireno, peakt tpcno 91 299.98 2 dist 90.2176min T, max_T3.44447 325.461
Begin processing the 30th record. run: 29097 subRun: 1 event: 29277 at 26-Sep-2024 16:31:51 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 54587 traces, 54587 summary
Retagger: tagging trace set: gauss with 42143 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 42143 traces tagged "gauss"
FrameSaver: q=928165 n=823115 tag=gauss
wclsFrameSaver: saving 54587 traces tagged "wiener"
FrameSaver: q=1.00779e+06 n=762478 tag=wiener
2403 hits in TPC 0
2375 hits in TPC 1
3635 hits in TPC 2
2405 hits in TPC 3
4657 hits in TPC 4
6273 hits in TPC 5
6018 hits in TPC 6
2994 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 123 XUs and 128 XVs -> 0 XUVs
C:0 T:2 22289 XUs and 10192 XVs -> 587 XUVs
C:0 T:3 199 XUs and 178 XVs -> 9 XUVs
C:0 T:4 417 XUs and 318 XVs -> 7 XUVs
C:0 T:5 17878 XUs and 21928 XVs -> 660 XUVs
C:0 T:6 32121 XUs and 16752 XVs -> 1072 XUVs
C:0 T:7 94 XUs and 86 XVs -> 1 XUVs
2336 XUVs total
1131 collection wire objects
2336 potential space points
Neighbour search...
63580 tests to find 36606 neighbours
Iterating with no regularization...
Begin: 9.90486e+06
0 9.42411e+06
1 9.39153e+06
2 9.38859e+06
Now with regularization...
Begin: 8.91512e+06
0 8.91418e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:34:14 BST Closed output file "np04hd_raw_run029097_0121_dataflow5_datawriter_0_20240906T124944_reco_stage1_reco_stage2_20240926T153414_keepup.root"
26-Sep-2024 16:34:14 BST Closed input file "np04hd_raw_run029097_0121_dataflow5_datawriter_0_20240906T124944_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 89.8631 135.618 218.885 128.433 28.5365 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000398 0.00266643 0.00708396 0.00287511 0.00126947 30
produce:fembfilter:PDHDFEMBFilter 0.733831 0.761297 0.818156 0.760794 0.0195867 30
produce:wclsdatahd:WireCellToolkit 53.5526 56.2604 58.6833 56.4729 1.30472 30
produce:gaushit:GausHitFinder 0.414084 0.77744 1.29593 0.721043 0.215934 30
produce:nhitsfilter:NumberOfHitsFilter 0.000277621 0.000424945 0.000691328 0.000404624 9.02981e-05 30
produce:reco3d:SpacePointSolver 0.206059 0.974916 3.19785 0.696102 0.716013 30
produce:hitpdune:DisambigFromSpacePoints 0.17295 0.592219 1.61531 0.452015 0.343451 30
produce:pandora:StandardPandora 4.19258 14.6477 38.4824 12.8134 7.92813 30
produce:pandoraWriter:StandardPandora 0.0928978 0.167951 0.282283 0.162863 0.0436568 30
produce:pandoraTrack:LArPandoraTrackCreation 0.600469 1.82546 4.62592 1.50988 0.940399 30
produce:pandoraShower:LArPandoraShowerCreation 0.503938 2.17715 7.25728 1.83581 1.45476 30
produce:pandoracalo:Calorimetry 0.323125 0.841816 1.96708 0.74143 0.371603 30
produce:pandoracalonosce:Calorimetry 0.288802 0.767602 1.84962 0.655996 0.359729 30
produce:pandorapid:Chi2ParticleID 0.00109199 0.00358623 0.053093 0.00179124 0.0092051 30
produce:pandoraShowercalo:ShowerCalorimetry 0.394528 2.02443 6.90792 1.69787 1.4623 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.391984 1.94277 6.72564 1.59442 1.40694 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0785034 0.133241 0.246458 0.125046 0.0358669 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0316081 0.0509558 0.0936261 0.0489024 0.013268 30
produce:emtrkmichelid:EmTrackMichelId 26.7968 49.8933 81.7139 47.6411 13.2261 30
[art]:TriggerResults:TriggerResultInserter 3.6353e-05 4.68204e-05 9.025e-05 4.55745e-05 9.38212e-06 30
end_path:out1:RootOutput 7.674e-06 1.26306e-05 4.5085e-05 1.028e-05 7.86585e-06 30
end_path:michelremoving:michelremoving 0.0253822 0.0974268 0.585178 0.0433303 0.113634 30
end_path:michelremovinggnocchi:michelremoving 0.0214438 0.0923775 0.586544 0.0371899 0.11481 30
end_path:calibanagnocchi:CalibAnaTree 0.186071 0.61581 1.45531 0.553332 0.3063 30
end_path:out1:RootOutput(write) 0.463936 0.821991 1.43981 0.767263 0.217802 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4975 MB
Peak resident set size usage (VmHWM): 2300.76 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0121_dataflow5_datawriter_0_20240906T124944_reco_stage1_reco_stage2_20240926T153414_keepup.root
\tHists: np04hd_raw_run029097_0121_dataflow5_datawriter_0_20240906T124944_reco_stage1_reco_stage2_20240926T153414_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
