Jobsub ID 263757.133@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263757.133@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:33 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce3 |
| Last heartbeat | 2024-09-26 15:20:46 |
| From worker node | Hostname | n182.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:15 |
| Input files | hd-protodune:np04hd_raw_run029094_0156_dataflow1_datawriter_0_20240906T094943.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:20:46 |
| Saved logs | justin-logs:263757.133-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
XUs and 16177 XVs -> 1736 XUVs
C:0 T:6 12791 XUs and 12200 XVs -> 692 XUVs
C:0 T:7 185 XUs and 320 XVs -> 19 XUVs
3302 XUVs total
1473 collection wire objects
3302 potential space points
Neighbour search...
143444 tests to find 75502 neighbours
Iterating with no regularization...
Begin: 1.31495e+07
0 1.18846e+07
1 1.17636e+07
2 1.17504e+07
3 1.17463e+07
Now with regularization...
Begin: 1.10618e+07
0 1.1059e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29094 subRun: 1 event: 37665 at 26-Sep-2024 16:16:49 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 57333 traces, 57333 summary
Retagger: tagging trace set: gauss with 41522 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41522 traces tagged "gauss"
FrameSaver: q=988075 n=730868 tag=gauss
wclsFrameSaver: saving 57333 traces tagged "wiener"
FrameSaver: q=1.0574e+06 n=648280 tag=wiener
1817 hits in TPC 0
1465 hits in TPC 1
4760 hits in TPC 2
3651 hits in TPC 3
1640 hits in TPC 4
2702 hits in TPC 5
6519 hits in TPC 6
5564 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2748 XUs and 1752 XVs -> 28 XUVs
C:0 T:2 9554 XUs and 11690 XVs -> 590 XUVs
C:0 T:3 170 XUs and 194 XVs -> 9 XUVs
C:0 T:4 31 XUs and 47 XVs -> 5 XUVs
C:0 T:5 3223 XUs and 4351 XVs -> 138 XUVs
C:0 T:6 26296 XUs and 20106 XVs -> 985 XUVs
C:0 T:7 142 XUs and 277 XVs -> 14 XUVs
1769 XUVs total
1006 collection wire objects
1769 potential space points
Neighbour search...
18763 tests to find 11456 neighbours
Iterating with no regularization...
Begin: 1.07001e+07
0 1.0379e+07
1 1.03658e+07
2 1.03647e+07
Now with regularization...
Begin: 9.69198e+06
0 9.69142e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 37665 TrkackID 14 startx, y, z 100.605 565.214 460.84 wireno, peakt tpcno 1 3159.68 5 dist 0.605952min T, max_T2203.19 3485.01
no of hits closeby 1 event 37665 TrkackID 14 startx, y, z 100.605 565.214 460.84 wireno, peakt tpcno 1 3159.68 5 dist 0.605952min T, max_T2203.19 3485.01
Begin processing the 30th record. run: 29094 subRun: 1 event: 37673 at 26-Sep-2024 16:18:38 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 85098 traces, 85098 summary
Retagger: tagging trace set: gauss with 60245 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 60245 traces tagged "gauss"
FrameSaver: q=1.26256e+06 n=1032384 tag=gauss
wclsFrameSaver: saving 85098 traces tagged "wiener"
FrameSaver: q=1.37094e+06 n=915817 tag=wiener
1651 hits in TPC 0
2230 hits in TPC 1
8470 hits in TPC 2
5572 hits in TPC 3
2736 hits in TPC 4
5709 hits in TPC 5
9000 hits in TPC 6
6054 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3897 XUs and 8257 XVs -> 687 XUVs
C:0 T:2 37146 XUs and 50317 XVs -> 1274 XUVs
C:0 T:3 999 XUs and 993 XVs -> 45 XUVs
C:0 T:4 69 XUs and 195 XVs -> 7 XUVs
C:0 T:5 21069 XUs and 15342 XVs -> 368 XUVs
C:0 T:6 31148 XUs and 53149 XVs -> 1008 XUVs
C:0 T:7 458 XUs and 518 XVs -> 12 XUVs
3401 XUVs total
1639 collection wire objects
3401 potential space points
Neighbour search...
125321 tests to find 75300 neighbours
Iterating with no regularization...
Begin: 6.49445e+06
0 5.89643e+06
1 5.86037e+06
2 5.85668e+06
Now with regularization...
Begin: 5.57029e+06
0 5.56934e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 6 event 37673 TrkackID 6 startx, y, z 90.5579 598.341 99.656 wireno, peakt tpcno 206 4962.1 5 dist 2.12594min T, max_T4198.28 5349.75
no of hits closeby 6 event 37673 TrkackID 6 startx, y, z 90.5579 598.341 99.656 wireno, peakt tpcno 206 4962.1 5 dist 2.12594min T, max_T4198.28 5349.75
26-Sep-2024 16:20:20 BST Closed output file "np04hd_raw_run029094_0156_dataflow1_datawriter_0_20240906T094943_reco_stage1_reco_stage2_20240926T152020_keepup.root"
26-Sep-2024 16:20:20 BST Closed input file "np04hd_raw_run029094_0156_dataflow1_datawriter_0_20240906T094943_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 73.6102 139.114 1114.75 101.222 182.265 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000332853 0.00130561 0.00365659 0.0010347 0.000963597 30
produce:fembfilter:PDHDFEMBFilter 0.696321 0.875034 1.10182 0.784813 0.173256 30
produce:wclsdatahd:WireCellToolkit 44.5755 55.5006 68.3788 55.1253 8.69923 30
produce:gaushit:GausHitFinder 0.542466 0.955064 1.79863 0.888805 0.296101 30
produce:nhitsfilter:NumberOfHitsFilter 0.000286982 0.000421177 0.000651801 0.0004006 9.46495e-05 30
produce:reco3d:SpacePointSolver 0.224265 0.842774 3.41961 0.696636 0.627043 30
produce:hitpdune:DisambigFromSpacePoints 0.272643 0.760238 2.96685 0.60219 0.554221 30
produce:pandora:StandardPandora 7.81339 45.5539 954.247 13.8632 168.819 30
produce:pandoraWriter:StandardPandora 0.118922 0.193241 0.31544 0.180793 0.0574182 30
produce:pandoraTrack:LArPandoraTrackCreation 0.799946 2.60911 13.9712 2.15483 2.31657 30
produce:pandoraShower:LArPandoraShowerCreation 0.895049 2.72413 12.5007 2.1021 2.20633 30
produce:pandoracalo:Calorimetry 0.405619 1.04525 5.05327 0.917063 0.810341 30
produce:pandoracalonosce:Calorimetry 0.363429 0.977616 4.86193 0.828562 0.786616 30
produce:pandorapid:Chi2ParticleID 0.00143087 0.0022688 0.00428569 0.00202402 0.000669224 30
produce:pandoraShowercalo:ShowerCalorimetry 0.59144 2.19415 8.59693 1.66541 1.51278 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.584814 2.16043 7.86463 1.68235 1.43812 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0893244 0.166334 0.277177 0.157181 0.0553595 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0376937 0.0664613 0.107571 0.0631921 0.0226766 30
produce:emtrkmichelid:EmTrackMichelId 12.2685 20.3195 34.6029 19.8007 4.90768 30
[art]:TriggerResults:TriggerResultInserter 2.6665e-05 4.39397e-05 0.000118319 4.2337e-05 1.60127e-05 30
end_path:out1:RootOutput 6.409e-06 1.21382e-05 4.1236e-05 9.7155e-06 7.00141e-06 30
end_path:michelremoving:michelremoving 0.0266993 0.105878 0.409921 0.0627415 0.0923395 30
end_path:michelremovinggnocchi:michelremoving 0.0235686 0.100447 0.406748 0.0516608 0.0917699 30
end_path:calibanagnocchi:CalibAnaTree 0.185185 0.775401 3.38055 0.701556 0.559952 30
end_path:out1:RootOutput(write) 0.538612 0.991208 1.51571 1.00957 0.280702 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5775.88 MB
Peak resident set size usage (VmHWM): 2792.86 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0156_dataflow1_datawriter_0_20240906T094943_reco_stage1_reco_stage2_20240926T152020_keepup.root
\tHists: np04hd_raw_run029094_0156_dataflow1_datawriter_0_20240906T094943_reco_stage1_reco_stage2_20240926T152020_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
