Jobsub ID 78497.104@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78497.104@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 10:12:31 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 12:39:36 |
| From worker node | Hostname | wn030.hep |
| cpuinfo | Intel(R) Xeon(R) Gold 6226 CPU @ 2.70GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:46:35 |
| Input files | hd-protodune:np04hd_raw_run029081_0781_dataflow5_datawriter_0_20240905T051406.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 12:39:36 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
UVs
C:0 T:6 79436 XUs and 101970 XVs -> 4943 XUVs
C:0 T:7 978 XUs and 778 XVs -> 23 XUVs
8796 XUVs total
2763 collection wire objects
8796 potential space points
Neighbour search...
679986 tests to find 336274 neighbours
Iterating with no regularization...
Begin: 2.47191e+07
0 2.22828e+07
1 2.21544e+07
2 2.21385e+07
Now with regularization...
Begin: 2.09286e+07
0 2.09189e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 187669 at 26-Sep-2024 12:51:26 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83729 traces, 83729 summary
Retagger: tagging trace set: gauss with 61812 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 61812 traces tagged "gauss"
FrameSaver: q=1.3445e+06 n=1113553 tag=gauss
wclsFrameSaver: saving 83729 traces tagged "wiener"
FrameSaver: q=1.43521e+06 n=993189 tag=wiener
4871 hits in TPC 0
3777 hits in TPC 1
8560 hits in TPC 2
6873 hits in TPC 3
2582 hits in TPC 4
3635 hits in TPC 5
8069 hits in TPC 6
4320 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 370 XUs and 223 XVs -> 18 XUVs
C:0 T:2 51280 XUs and 42881 XVs -> 2372 XUVs
C:0 T:3 462 XUs and 284 XVs -> 15 XUVs
C:0 T:4 298 XUs and 266 XVs -> 19 XUVs
C:0 T:5 13764 XUs and 7086 XVs -> 319 XUVs
C:0 T:6 24327 XUs and 24798 XVs -> 542 XUVs
C:0 T:7 204 XUs and 155 XVs -> 5 XUVs
3290 XUVs total
1361 collection wire objects
3290 potential space points
Neighbour search...
151008 tests to find 90242 neighbours
Iterating with no regularization...
Begin: 8.7742e+06
0 7.8336e+06
1 7.75773e+06
2 7.75085e+06
Now with regularization...
Begin: 7.13872e+06
0 7.1356e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 187677 at 26-Sep-2024 12:53:26 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65181 traces, 65181 summary
Retagger: tagging trace set: gauss with 50065 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50065 traces tagged "gauss"
FrameSaver: q=1.13848e+06 n=968254 tag=gauss
wclsFrameSaver: saving 65181 traces tagged "wiener"
FrameSaver: q=1.22149e+06 n=881418 tag=wiener
3913 hits in TPC 0
2611 hits in TPC 1
3688 hits in TPC 2
2486 hits in TPC 3
4822 hits in TPC 4
8322 hits in TPC 5
5624 hits in TPC 6
3856 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 606 XUs and 406 XVs -> 47 XUVs
C:0 T:2 36846 XUs and 9160 XVs -> 424 XUVs
C:0 T:3 8 XUs and 14 XVs -> 0 XUVs
C:0 T:4 341 XUs and 362 XVs -> 17 XUVs
C:0 T:5 19581 XUs and 25793 XVs -> 968 XUVs
C:0 T:6 26211 XUs and 11555 XVs -> 456 XUVs
C:0 T:7 26 XUs and 40 XVs -> 2 XUVs
1914 XUVs total
1217 collection wire objects
1914 potential space points
Neighbour search...
27990 tests to find 16658 neighbours
Iterating with no regularization...
Begin: 1.54053e+07
0 1.48007e+07
1 1.47727e+07
2 1.47692e+07
Now with regularization...
Begin: 1.43001e+07
0 1.42991e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 187677 TrkackID 12 startx, y, z 132.129 513.188 65.9816 wireno, peakt tpcno 81 1656.76 2 dist 130.049min T, max_T5 1684.08
no of hits closeby 0 event 187677 TrkackID 12 startx, y, z 132.129 513.188 65.9816 wireno, peakt tpcno 81 1656.76 2 dist 130.049min T, max_T5 1684.08
26-Sep-2024 12:55:02 BST Closed output file "np04hd_raw_run029081_0781_dataflow5_datawriter_0_20240905T051406_reco_stage1_reco_stage2_20240926T115502_keepup.root"
26-Sep-2024 12:55:02 BST Closed input file "np04hd_raw_run029081_0781_dataflow5_datawriter_0_20240905T051406_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 50.7956 82.5875 119.792 80.369 14.0134 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000349793 0.00139497 0.00394734 0.00127791 0.000954359 30
produce:fembfilter:PDHDFEMBFilter 0.596467 0.648065 1.11127 0.607106 0.123898 30
produce:wclsdatahd:WireCellToolkit 44.1625 46.979 50.8622 46.8284 1.36609 30
produce:gaushit:GausHitFinder 0.420251 0.627797 1.26731 0.601487 0.158856 30
produce:nhitsfilter:NumberOfHitsFilter 0.000298143 0.000388143 0.00058794 0.000357528 7.67694e-05 30
produce:reco3d:SpacePointSolver 0.190985 0.744055 2.48708 0.635493 0.481252 29
produce:hitpdune:DisambigFromSpacePoints 0.207858 0.410223 0.964969 0.363116 0.174984 29
produce:pandora:StandardPandora 5.11138 10.2907 23.3072 9.31318 3.86618 29
produce:pandoraWriter:StandardPandora 0.102017 0.14809 0.257065 0.146921 0.0340134 29
produce:pandoraTrack:LArPandoraTrackCreation 0.923433 2.10866 5.90316 1.73382 1.13952 29
produce:pandoraShower:LArPandoraShowerCreation 0.962224 2.41314 7.78236 2.13571 1.46827 29
produce:pandoracalo:Calorimetry 0.420329 0.882918 2.52215 0.713879 0.449377 29
produce:pandoracalonosce:Calorimetry 0.407073 0.815275 2.55163 0.633576 0.450491 29
produce:pandorapid:Chi2ParticleID 0.00114092 0.00166368 0.0032529 0.00151262 0.00047656 29
produce:pandoraShowercalo:ShowerCalorimetry 0.762757 2.12902 6.929 1.73949 1.31724 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.72693 2.07094 6.49833 1.5557 1.28095 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0803014 0.110989 0.210695 0.10555 0.0285409 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0306447 0.0442518 0.0702281 0.0418521 0.0098464 29
produce:emtrkmichelid:EmTrackMichelId 8.4608 11.7377 17.5014 11.7424 2.17086 29
[art]:TriggerResults:TriggerResultInserter 2.2852e-05 3.85033e-05 6.8912e-05 3.77715e-05 9.12357e-06 30
end_path:out1:RootOutput 6.716e-06 1.05931e-05 2.0377e-05 1.02905e-05 2.65715e-06 30
end_path:michelremoving:michelremoving 3.7765e-05 0.0676219 0.18625 0.0487877 0.0461815 30
end_path:michelremovinggnocchi:michelremoving 2.3302e-05 0.0640696 0.187309 0.0447312 0.0456572 30
end_path:calibanagnocchi:CalibAnaTree 2.4665e-05 0.589693 1.28719 0.557214 0.277361 30
end_path:out1:RootOutput(write) 0.309345 0.630661 1.04012 0.630482 0.14576 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5776.21 MB
Peak resident set size usage (VmHWM): 2606.84 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0781_dataflow5_datawriter_0_20240905T051406_reco_stage1_reco_stage2_20240926T115502_keepup.root
\tHists: np04hd_raw_run029081_0781_dataflow5_datawriter_0_20240905T051406_reco_stage1_reco_stage2_20240926T115502_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
