Jobsub ID 79091.0@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 79091.0@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3460 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:25:17 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce01 |
| Last heartbeat | 2024-09-26 15:37:45 |
| From worker node | Hostname | wn2205251.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7513 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:26:48 |
| Input files | hd-protodune:np04hd_raw_run029218_0098_dataflow1_datawriter_0_20240917T165220.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:37:45 |
| Saved logs | justin-logs:79091.0-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
C:0 T:1 378 XUs and 432 XVs -> 44 XUVs
C:0 T:2 13819 XUs and 9037 XVs -> 205 XUVs
C:0 T:3 2 XUs and 2 XVs -> 0 XUVs
C:0 T:4 247 XUs and 212 XVs -> 10 XUVs
C:0 T:5 23399 XUs and 18907 XVs -> 626 XUVs
C:0 T:6 11767 XUs and 11152 XVs -> 398 XUVs
C:0 T:7 42 XUs and 27 XVs -> 0 XUVs
1283 XUVs total
992 collection wire objects
1283 potential space points
Neighbour search...
13817 tests to find 7640 neighbours
Iterating with no regularization...
Begin: 5.53859e+06
0 5.3706e+06
1 5.36247e+06
2 5.36183e+06
Now with regularization...
Begin: 5.04089e+06
0 5.04073e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 24529 TrkackID 1 startx, y, z 245.969 606.831 308.626 wireno, peakt tpcno 0 815 5 dist 79.9774min T, max_T12.7922 3553.07
no of hits closeby 0 event 24529 TrkackID 1 startx, y, z 245.969 606.831 308.626 wireno, peakt tpcno 0 815 5 dist 79.9774min T, max_T12.7922 3553.07
Begin processing the 30th record. run: 29218 subRun: 1 event: 24537 at 26-Sep-2024 16:34:44 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 55262 traces, 55262 summary
Retagger: tagging trace set: gauss with 40521 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40521 traces tagged "gauss"
FrameSaver: q=926594 n=789259 tag=gauss
wclsFrameSaver: saving 55262 traces tagged "wiener"
FrameSaver: q=991402 n=717657 tag=wiener
1817 hits in TPC 0
1495 hits in TPC 1
6393 hits in TPC 2
4360 hits in TPC 3
5230 hits in TPC 4
6464 hits in TPC 5
2635 hits in TPC 6
1750 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 29 XUs and 10 XVs -> 1 XUVs
C:0 T:2 21588 XUs and 22715 XVs -> 1325 XUVs
C:0 T:3 155 XUs and 230 XVs -> 7 XUVs
C:0 T:4 231 XUs and 323 XVs -> 11 XUVs
C:0 T:5 16949 XUs and 13273 XVs -> 602 XUVs
C:0 T:6 4359 XUs and 5984 XVs -> 177 XUVs
C:0 T:7 94 XUs and 87 XVs -> 7 XUVs
2130 XUVs total
1207 collection wire objects
2130 potential space points
Neighbour search...
32942 tests to find 19560 neighbours
Iterating with no regularization...
Begin: 6.77033e+06
0 6.38142e+06
1 6.34688e+06
2 6.34429e+06
Now with regularization...
Begin: 5.78123e+06
0 5.78057e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 31st record. run: 29218 subRun: 1 event: 24545 at 26-Sep-2024 16:36:03 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66457 traces, 66457 summary
Retagger: tagging trace set: gauss with 47462 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47462 traces tagged "gauss"
FrameSaver: q=1.01844e+06 n=813843 tag=gauss
wclsFrameSaver: saving 66457 traces tagged "wiener"
FrameSaver: q=1.10534e+06 n=722611 tag=wiener
2133 hits in TPC 0
3117 hits in TPC 1
7800 hits in TPC 2
3987 hits in TPC 3
3447 hits in TPC 4
7758 hits in TPC 5
3547 hits in TPC 6
1073 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 8 XUs and 19 XVs -> 1 XUVs
C:0 T:1 3077 XUs and 3378 XVs -> 257 XUVs
C:0 T:2 22436 XUs and 21992 XVs -> 695 XUVs
C:0 T:3 510 XUs and 576 XVs -> 9 XUVs
C:0 T:4 207 XUs and 180 XVs -> 5 XUVs
C:0 T:5 42238 XUs and 24134 XVs -> 751 XUVs
C:0 T:6 6891 XUs and 6574 XVs -> 194 XUVs
C:0 T:7 2 XUs and 2 XVs -> 0 XUVs
1912 XUVs total
1151 collection wire objects
1912 potential space points
Neighbour search...
30544 tests to find 16694 neighbours
Iterating with no regularization...
Begin: 6.75075e+06
0 6.4424e+06
1 6.43554e+06
2 6.43511e+06
Now with regularization...
Begin: 6.01633e+06
0 6.01602e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:37:19 BST Closed output file "np04hd_raw_run029218_0098_dataflow1_datawriter_0_20240917T165220_reco_stage1_reco_stage2_20240926T153719_keepup.root"
26-Sep-2024 16:37:19 BST Closed input file "np04hd_raw_run029218_0098_dataflow1_datawriter_0_20240917T165220_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 57.6819 92.8726 138.003 89.8748 18.18 31
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000521091 0.00303182 0.00551084 0.00303433 0.00116195 31
produce:fembfilter:PDHDFEMBFilter 0.523791 0.711003 0.798138 0.753676 0.0955024 31
produce:wclsdatahd:WireCellToolkit 31.0082 39.5928 46.4013 40.242 4.09407 31
produce:gaushit:GausHitFinder 0.322997 0.652297 1.21282 0.639368 0.182089 31
produce:nhitsfilter:NumberOfHitsFilter 0.000250712 0.000386639 0.000779699 0.000373173 9.97476e-05 31
produce:reco3d:SpacePointSolver 0.186282 0.790736 3.67087 0.554625 0.686059 31
produce:hitpdune:DisambigFromSpacePoints 0.0951509 0.483227 1.41429 0.413464 0.281073 31
produce:pandora:StandardPandora 4.79705 11.4157 27.6836 9.59175 5.40947 31
produce:pandoraWriter:StandardPandora 0.0890262 0.185509 0.297856 0.179506 0.0529252 31
produce:pandoraTrack:LArPandoraTrackCreation 0.604891 1.44102 3.28732 1.29246 0.641716 31
produce:pandoraShower:LArPandoraShowerCreation 0.370858 1.65347 3.21933 1.86872 0.799542 31
produce:pandoracalo:Calorimetry 0.313042 0.701083 1.12268 0.666417 0.248627 31
produce:pandoracalonosce:Calorimetry 0.275983 0.625301 1.07299 0.597372 0.232468 31
produce:pandorapid:Chi2ParticleID 0.00120006 0.0019493 0.00357674 0.00180637 0.000509939 31
produce:pandoraShowercalo:ShowerCalorimetry 0.343929 1.48293 3.63807 1.61137 0.739883 31
produce:pandoraShowercalonosce:ShowerCalorimetry 0.339523 1.4275 3.21012 1.44893 0.666426 31
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0709854 0.145689 0.271273 0.141184 0.0485608 31
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.021602 0.0492449 0.0854828 0.0473647 0.0163312 31
produce:emtrkmichelid:EmTrackMichelId 16.0801 29.5888 45.9225 27.5049 7.42385 31
[art]:TriggerResults:TriggerResultInserter 2.8874e-05 6.71722e-05 0.000123242 6.5904e-05 1.85277e-05 31
end_path:out1:RootOutput 7.594e-06 1.72907e-05 3.3524e-05 1.7182e-05 6.28307e-06 31
end_path:michelremoving:michelremoving 0.0183663 0.0988471 0.409994 0.0458403 0.101352 31
end_path:michelremovinggnocchi:michelremoving 0.0178586 0.0961914 0.415033 0.0374406 0.103723 31
end_path:calibanagnocchi:CalibAnaTree 0.169909 0.595866 1.30096 0.498267 0.293683 31
end_path:out1:RootOutput(write) 0.438713 0.935561 1.36609 0.915044 0.234789 31
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4930.58 MB
Peak resident set size usage (VmHWM): 2572.67 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029218_0098_dataflow1_datawriter_0_20240917T165220_reco_stage1_reco_stage2_20240926T153719_keepup.root
\tHists: np04hd_raw_run029218_0098_dataflow1_datawriter_0_20240917T165220_reco_stage1_reco_stage2_20240926T153719_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
