Jobsub ID 263785.7@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263785.7@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3453 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:27:25 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 14:28:03 |
| From worker node | Hostname | wn035.hep |
| cpuinfo | AMD EPYC 7352 24-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:27:42 |
| Input files | hd-protodune:np04hd_raw_run029107_3575_dataflow4_datawriter_0_20240909T033622.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:28:03 |
| Saved logs | justin-logs:263785.7-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
TPC 4
4775 hits in TPC 5
7617 hits in TPC 6
7208 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 25360 XUs and 48648 XVs -> 2307 XUVs
C:0 T:2 14164 XUs and 23272 XVs -> 1171 XUVs
C:0 T:3 793 XUs and 1423 XVs -> 36 XUVs
C:0 T:4 42 XUs and 39 XVs -> 1 XUVs
C:0 T:5 9434 XUs and 9622 XVs -> 514 XUVs
C:0 T:6 27873 XUs and 30331 XVs -> 1598 XUVs
C:0 T:7 642 XUs and 511 XVs -> 6 XUVs
5633 XUVs total
1560 collection wire objects
5633 potential space points
Neighbour search...
456021 tests to find 232298 neighbours
Iterating with no regularization...
Begin: 2.38352e+07
0 2.22478e+07
1 2.21347e+07
2 2.21238e+07
Now with regularization...
Begin: 2.06887e+07
0 2.06706e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 858252 at 26-Sep-2024 15:25:20 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 75818 traces, 75818 summary
Retagger: tagging trace set: gauss with 54264 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54264 traces tagged "gauss"
FrameSaver: q=1.11463e+06 n=970672 tag=gauss
wclsFrameSaver: saving 75818 traces tagged "wiener"
FrameSaver: q=1.22121e+06 n=877050 tag=wiener
3531 hits in TPC 0
2527 hits in TPC 1
7537 hits in TPC 2
2021 hits in TPC 3
2980 hits in TPC 4
5156 hits in TPC 5
7509 hits in TPC 6
6198 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1075 XUs and 499 XVs -> 12 XUVs
C:0 T:2 18674 XUs and 40953 XVs -> 362 XUVs
C:0 T:3 43 XUs and 69 XVs -> 2 XUVs
C:0 T:4 4 XUs and 8 XVs -> 0 XUVs
C:0 T:5 25882 XUs and 17950 XVs -> 800 XUVs
C:0 T:6 54147 XUs and 59782 XVs -> 3159 XUVs
C:0 T:7 35 XUs and 125 XVs -> 0 XUVs
4335 XUVs total
1438 collection wire objects
4335 potential space points
Neighbour search...
222423 tests to find 131898 neighbours
Iterating with no regularization...
Begin: 7.10305e+06
0 6.36771e+06
1 6.28712e+06
2 6.27297e+06
3 6.26871e+06
Now with regularization...
Begin: 5.67579e+06
0 5.67303e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 858260 at 26-Sep-2024 15:26:28 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 80977 traces, 80977 summary
Retagger: tagging trace set: gauss with 58819 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58819 traces tagged "gauss"
FrameSaver: q=1.33087e+06 n=1067920 tag=gauss
wclsFrameSaver: saving 80977 traces tagged "wiener"
FrameSaver: q=1.44483e+06 n=968304 tag=wiener
2335 hits in TPC 0
2921 hits in TPC 1
8403 hits in TPC 2
5298 hits in TPC 3
3010 hits in TPC 4
8421 hits in TPC 5
7497 hits in TPC 6
4339 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 31 XUs and 25 XVs -> 4 XUVs
C:0 T:2 39620 XUs and 48051 XVs -> 2108 XUVs
C:0 T:3 440 XUs and 644 XVs -> 17 XUVs
C:0 T:4 161 XUs and 247 XVs -> 17 XUVs
C:0 T:5 61988 XUs and 36120 XVs -> 1953 XUVs
C:0 T:6 29510 XUs and 28749 XVs -> 1647 XUVs
C:0 T:7 635 XUs and 636 XVs -> 11 XUVs
5757 XUVs total
2633 collection wire objects
5757 potential space points
Neighbour search...
181707 tests to find 104162 neighbours
Iterating with no regularization...
Begin: 1.24202e+07
0 1.07828e+07
1 1.06919e+07
2 1.06798e+07
3 1.06768e+07
Now with regularization...
Begin: 9.80037e+06
0 9.79871e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 858260 TrkackID 15 startx, y, z -91.6208 591.763 410.631 wireno, peakt tpcno 443 2419.05 6 dist 0min T, max_T1599.93 2799.09
no of hits closeby 1 event 858260 TrkackID 15 startx, y, z -91.6208 591.763 410.631 wireno, peakt tpcno 443 2419.05 6 dist 0min T, max_T1599.93 2799.09
26-Sep-2024 15:27:43 BST Closed output file "np04hd_raw_run029107_3575_dataflow4_datawriter_0_20240909T033622_reco_stage1_reco_stage2_20240926T142743_keepup.root"
26-Sep-2024 15:27:43 BST Closed input file "np04hd_raw_run029107_3575_dataflow4_datawriter_0_20240909T033622_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 42.4118 69.841 99.9664 69.8454 14.0224 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000309109 0.00124859 0.00239367 0.00110328 0.000545559 30
produce:fembfilter:PDHDFEMBFilter 0.510913 0.522031 0.603419 0.51633 0.0202421 30
produce:wclsdatahd:WireCellToolkit 27.5465 33.2338 39.5997 34.6002 3.56517 30
produce:gaushit:GausHitFinder 0.255603 0.538309 1.70237 0.502376 0.251115 30
produce:nhitsfilter:NumberOfHitsFilter 0.00017832 0.000311974 0.000794596 0.000289848 0.000118021 30
produce:reco3d:SpacePointSolver 0.0660273 0.810448 4.60663 0.626166 0.902361 29
produce:hitpdune:DisambigFromSpacePoints 0.0771718 0.359317 0.757642 0.311927 0.188632 29
produce:pandora:StandardPandora 2.66883 8.73931 16.62 8.64309 3.77937 29
produce:pandoraWriter:StandardPandora 0.048902 0.11187 0.174913 0.115391 0.0332933 29
produce:pandoraTrack:LArPandoraTrackCreation 0.232628 0.798594 1.72852 0.744022 0.332628 29
produce:pandoraShower:LArPandoraShowerCreation 0.246847 1.02527 2.45755 0.935732 0.546652 29
produce:pandoracalo:Calorimetry 0.134934 0.426742 0.812604 0.41837 0.161593 29
produce:pandoracalonosce:Calorimetry 0.117346 0.383384 0.698048 0.373428 0.147848 29
produce:pandorapid:Chi2ParticleID 0.000776111 0.00132921 0.00256519 0.00122732 0.000336363 29
produce:pandoraShowercalo:ShowerCalorimetry 0.213201 0.955835 2.47683 0.926529 0.521736 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.208505 0.927572 2.3407 0.893337 0.514956 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0453494 0.104245 0.178488 0.10033 0.0286819 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0166564 0.0363395 0.0604475 0.035825 0.00970602 29
produce:emtrkmichelid:EmTrackMichelId 11.1265 20.8651 32.571 21.3941 5.31311 29
[art]:TriggerResults:TriggerResultInserter 1.2554e-05 1.9535e-05 7.9702e-05 1.47935e-05 1.24927e-05 30
end_path:out1:RootOutput 3.356e-06 4.82417e-06 2.3675e-05 4.123e-06 3.55384e-06 30
end_path:michelremoving:michelremoving 1.8976e-05 0.0808157 0.415551 0.047492 0.0888332 30
end_path:michelremovinggnocchi:michelremoving 5.621e-06 0.0807148 0.421715 0.0340084 0.0926779 30
end_path:calibanagnocchi:CalibAnaTree 5e-06 0.315965 0.61389 0.318328 0.148762 30
end_path:out1:RootOutput(write) 0.298439 0.619984 0.934767 0.637287 0.155154 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4970.93 MB
Peak resident set size usage (VmHWM): 2371.76 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_3575_dataflow4_datawriter_0_20240909T033622_reco_stage1_reco_stage2_20240926T142743_keepup.root
\tHists: np04hd_raw_run029107_3575_dataflow4_datawriter_0_20240909T033622_reco_stage1_reco_stage2_20240926T142743_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
