Jobsub ID 263757.131@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263757.131@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:33 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 15:05:47 |
| From worker node | Hostname | n224.dur.scotgrid.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5220 CPU @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:28 |
| Input files | hd-protodune:np04hd_raw_run029097_0076_dataflow3_datawriter_0_20240906T115345.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:05:47 |
| Saved logs | justin-logs:263757.131-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
and 1111 XVs -> 58 XUVs
C:0 T:2 25448 XUs and 38839 XVs -> 1855 XUVs
C:0 T:3 104 XUs and 129 XVs -> 1 XUVs
C:0 T:4 48 XUs and 64 XVs -> 1 XUVs
C:0 T:5 8243 XUs and 24352 XVs -> 1427 XUVs
C:0 T:6 71288 XUs and 64550 XVs -> 3869 XUVs
C:0 T:7 182 XUs and 215 XVs -> 9 XUVs
7220 XUVs total
1875 collection wire objects
7220 potential space points
Neighbour search...
596300 tests to find 290610 neighbours
Iterating with no regularization...
Begin: 2.90158e+07
0 2.69207e+07
1 2.67988e+07
2 2.67807e+07
Now with regularization...
Begin: 2.61112e+07
0 2.61052e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 18467 at 26-Sep-2024 16:02:23 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 53493 traces, 53493 summary
Retagger: tagging trace set: gauss with 41883 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41883 traces tagged "gauss"
FrameSaver: q=938453 n=824062 tag=gauss
wclsFrameSaver: saving 53493 traces tagged "wiener"
FrameSaver: q=979456 n=744472 tag=wiener
4660 hits in TPC 0
3999 hits in TPC 1
4790 hits in TPC 2
4087 hits in TPC 3
1825 hits in TPC 4
3080 hits in TPC 5
3873 hits in TPC 6
1855 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 165 XUs and 115 XVs -> 5 XUVs
C:0 T:2 9914 XUs and 9301 XVs -> 577 XUVs
C:0 T:3 123 XUs and 149 XVs -> 8 XUVs
C:0 T:4 109 XUs and 96 XVs -> 17 XUVs
C:0 T:5 6262 XUs and 3383 XVs -> 49 XUVs
C:0 T:6 8348 XUs and 9249 XVs -> 212 XUVs
C:0 T:7 224 XUs and 297 XVs -> 18 XUVs
886 XUVs total
659 collection wire objects
886 potential space points
Neighbour search...
8220 tests to find 4580 neighbours
Iterating with no regularization...
Begin: 4.86165e+06
0 4.74499e+06
1 4.74047e+06
Now with regularization...
Begin: 4.39419e+06
0 4.39361e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 10 event 18467 TrkackID 7 startx, y, z -87.8388 593.943 349.421 wireno, peakt tpcno 443 4575.61 6 dist 12.5332min T, max_T3905.17 5029.11
no of hits closeby 10 event 18467 TrkackID 7 startx, y, z -87.8388 593.943 349.421 wireno, peakt tpcno 443 4575.61 6 dist 12.5332min T, max_T3905.17 5029.11
Begin processing the 30th record. run: 29097 subRun: 1 event: 18475 at 26-Sep-2024 16:03:51 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 60703 traces, 60703 summary
Retagger: tagging trace set: gauss with 43359 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43359 traces tagged "gauss"
FrameSaver: q=1.00085e+06 n=802091 tag=gauss
wclsFrameSaver: saving 60703 traces tagged "wiener"
FrameSaver: q=1.0857e+06 n=725229 tag=wiener
3670 hits in TPC 0
3609 hits in TPC 1
3774 hits in TPC 2
3251 hits in TPC 3
3828 hits in TPC 4
6130 hits in TPC 5
4308 hits in TPC 6
2060 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 8 XUs and 4 XVs -> 0 XUVs
C:0 T:1 950 XUs and 465 XVs -> 65 XUVs
C:0 T:2 11193 XUs and 27313 XVs -> 2632 XUVs
C:0 T:3 27 XUs and 239 XVs -> 0 XUVs
C:0 T:4 114 XUs and 75 XVs -> 3 XUVs
C:0 T:5 18167 XUs and 18052 XVs -> 1252 XUVs
C:0 T:6 9499 XUs and 6342 XVs -> 321 XUVs
C:0 T:7 11 XUs and 3 XVs -> 0 XUVs
4273 XUVs total
1550 collection wire objects
4273 potential space points
Neighbour search...
155537 tests to find 90776 neighbours
Iterating with no regularization...
Begin: 9.04075e+06
0 8.22415e+06
1 8.15676e+06
2 8.1476e+06
3 8.1453e+06
Now with regularization...
Begin: 7.31491e+06
0 7.31326e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:05:17 BST Closed output file "np04hd_raw_run029097_0076_dataflow3_datawriter_0_20240906T115345_reco_stage1_reco_stage2_20240926T150517_keepup.root"
26-Sep-2024 16:05:17 BST Closed input file "np04hd_raw_run029097_0076_dataflow3_datawriter_0_20240906T115345_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 70.1795 102.88 152.885 102.676 20.8643 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000386182 0.00136574 0.00433729 0.00115212 0.000915878 30
produce:fembfilter:PDHDFEMBFilter 0.691132 0.836433 1.18498 0.710655 0.179405 30
produce:wclsdatahd:WireCellToolkit 43.4956 51.7312 64.9672 47.1095 8.32355 30
produce:gaushit:GausHitFinder 0.478747 0.884042 1.51742 0.81719 0.259318 30
produce:nhitsfilter:NumberOfHitsFilter 0.000258662 0.000414134 0.000833545 0.000395697 0.000109473 30
produce:reco3d:SpacePointSolver 0.197508 1.0254 2.91961 0.754669 0.734957 30
produce:hitpdune:DisambigFromSpacePoints 0.177751 0.667461 1.40601 0.60072 0.335266 30
produce:pandora:StandardPandora 5.31374 14.6834 34.8576 13.899 6.80357 30
produce:pandoraWriter:StandardPandora 0.107636 0.179735 0.283879 0.175442 0.0456022 30
produce:pandoraTrack:LArPandoraTrackCreation 0.713534 2.09101 4.59968 1.7136 1.08881 30
produce:pandoraShower:LArPandoraShowerCreation 0.464713 2.26154 5.59506 1.73009 1.33581 30
produce:pandoracalo:Calorimetry 0.35336 0.876077 1.91473 0.743555 0.401788 30
produce:pandoracalonosce:Calorimetry 0.313398 0.802982 1.69471 0.659838 0.373623 30
produce:pandorapid:Chi2ParticleID 0.00126969 0.00216872 0.00369582 0.00208021 0.000601915 30
produce:pandoraShowercalo:ShowerCalorimetry 0.356829 1.93713 5.20612 1.52117 1.14709 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.342802 1.90043 4.70731 1.46068 1.10578 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0887928 0.159314 0.299446 0.153728 0.0485372 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0332614 0.0614834 0.114193 0.0590978 0.0187263 30
produce:emtrkmichelid:EmTrackMichelId 10.8759 20.7249 27.9825 20.1953 4.42927 30
[art]:TriggerResults:TriggerResultInserter 2.6442e-05 9.1321e-05 0.000974424 4.41825e-05 0.000184765 30
end_path:out1:RootOutput 7.356e-06 1.71068e-05 0.00014649 1.1286e-05 2.45551e-05 30
end_path:michelremoving:michelremoving 0.0259056 0.108373 0.472596 0.0556166 0.110744 30
end_path:michelremovinggnocchi:michelremoving 0.0237119 0.0952527 0.472974 0.0529704 0.106813 30
end_path:calibanagnocchi:CalibAnaTree 0.226253 0.694602 1.91137 0.615538 0.365266 30
end_path:out1:RootOutput(write) 0.514056 0.957621 1.65839 0.991832 0.275606 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5806.04 MB
Peak resident set size usage (VmHWM): 2844.7 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0076_dataflow3_datawriter_0_20240906T115345_reco_stage1_reco_stage2_20240926T150517_keepup.root
\tHists: np04hd_raw_run029097_0076_dataflow3_datawriter_0_20240906T115345_reco_stage1_reco_stage2_20240926T150517_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
