Jobsub ID 263757.126@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263757.126@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:33 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:37:03 |
| From worker node | Hostname | n249.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:22 |
| Input files | hd-protodune:np04hd_raw_run029097_0072_dataflow1_datawriter_0_20240906T114904.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:37:01 |
| Saved logs | justin-logs:263757.126-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
8e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 17497 TrkackID 1 startx, y, z 241.422 605.76 361.939 wireno, peakt tpcno 0 2978.99 5 dist 15.8747min T, max_T1918.31 4993
no of hits closeby 0 event 17497 TrkackID 1 startx, y, z 241.422 605.76 361.939 wireno, peakt tpcno 0 2978.99 5 dist 15.8747min T, max_T1918.31 4993
Begin processing the 29th record. run: 29097 subRun: 1 event: 17505 at 26-Sep-2024 15:32:27 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 68937 traces, 68937 summary
Retagger: tagging trace set: gauss with 54493 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54493 traces tagged "gauss"
FrameSaver: q=1.40187e+06 n=1144323 tag=gauss
wclsFrameSaver: saving 68937 traces tagged "wiener"
FrameSaver: q=1.51025e+06 n=1063919 tag=wiener
4109 hits in TPC 0
3512 hits in TPC 1
14387 hits in TPC 2
9894 hits in TPC 3
2225 hits in TPC 4
4791 hits in TPC 5
3166 hits in TPC 6
3351 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 55 XUs and 77 XVs -> 3 XUVs
C:0 T:2 112042 XUs and 129233 XVs -> 8385 XUVs
C:0 T:3 14420 XUs and 20554 XVs -> 958 XUVs
C:0 T:4 38 XUs and 42 XVs -> 9 XUVs
C:0 T:5 10632 XUs and 12671 XVs -> 249 XUVs
C:0 T:6 5807 XUs and 8053 XVs -> 407 XUVs
C:0 T:7 875 XUs and 1201 XVs -> 55 XUVs
10066 XUVs total
3890 collection wire objects
10066 potential space points
Neighbour search...
999076 tests to find 255774 neighbours
Iterating with no regularization...
Begin: 2.5545e+07
0 1.92123e+07
1 1.86425e+07
2 1.85675e+07
3 1.85516e+07
Now with regularization...
Begin: 1.70419e+07
0 1.70203e+07
1 1.70151e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 17505 TrkackID 5 startx, y, z -78.0465 590.272 313.99 wireno, peakt tpcno 399 2987.03 6 dist 0.0196881min T, max_T2982.94 4264.67
no of hits closeby 1 event 17505 TrkackID 5 startx, y, z -78.0465 590.272 313.99 wireno, peakt tpcno 399 2987.03 6 dist 0.0196881min T, max_T2982.94 4264.67
Begin processing the 30th record. run: 29097 subRun: 1 event: 17513 at 26-Sep-2024 15:34:18 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 52173 traces, 52173 summary
Retagger: tagging trace set: gauss with 41031 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41031 traces tagged "gauss"
FrameSaver: q=995961 n=841422 tag=gauss
wclsFrameSaver: saving 52173 traces tagged "wiener"
FrameSaver: q=1.05745e+06 n=773821 tag=wiener
1750 hits in TPC 0
2541 hits in TPC 1
4588 hits in TPC 2
2178 hits in TPC 3
3400 hits in TPC 4
3723 hits in TPC 5
7146 hits in TPC 6
5329 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 5 XUs and 5 XVs -> 0 XUVs
C:0 T:2 10511 XUs and 7970 XVs -> 562 XUVs
C:0 T:3 504 XUs and 804 XVs -> 18 XUVs
C:0 T:4 310 XUs and 579 XVs -> 4 XUVs
C:0 T:5 6415 XUs and 8780 XVs -> 574 XUVs
C:0 T:6 25385 XUs and 30042 XVs -> 1196 XUVs
C:0 T:7 36 XUs and 57 XVs -> 1 XUVs
2355 XUVs total
1449 collection wire objects
2355 potential space points
Neighbour search...
54107 tests to find 25858 neighbours
Iterating with no regularization...
Begin: 8.48628e+06
0 7.83701e+06
1 7.80012e+06
2 7.79592e+06
Now with regularization...
Begin: 7.09585e+06
0 7.09395e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 17513 TrkackID 2 startx, y, z 149.145 599.809 262.544 wireno, peakt tpcno 443 1323.81 5 dist 0min T, max_T542.609 2506.53
no of hits closeby 2 event 17513 TrkackID 2 startx, y, z 149.145 599.809 262.544 wireno, peakt tpcno 443 1323.81 5 dist 0min T, max_T542.609 2506.53
26-Sep-2024 15:35:20 BST Closed output file "np04hd_raw_run029097_0072_dataflow1_datawriter_0_20240906T114904_reco_stage1_reco_stage2_20240926T143520_keepup.root"
26-Sep-2024 15:35:20 BST Closed input file "np04hd_raw_run029097_0072_dataflow1_datawriter_0_20240906T114904_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 44.2434 66.4775 111.262 63.419 14.8004 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000320528 0.00114047 0.00212532 0.00115627 0.00049537 30
produce:fembfilter:PDHDFEMBFilter 0.442954 0.52862 0.692472 0.456155 0.104118 30
produce:wclsdatahd:WireCellToolkit 24.5905 28.758 34.8785 27.1351 3.75042 30
produce:gaushit:GausHitFinder 0.292032 0.561001 0.978043 0.514896 0.193194 30
produce:nhitsfilter:NumberOfHitsFilter 0.000138523 0.000257558 0.000400619 0.000253596 5.61181e-05 30
produce:reco3d:SpacePointSolver 0.093342 0.720611 2.90484 0.453119 0.670567 30
produce:hitpdune:DisambigFromSpacePoints 0.120548 0.424165 0.98502 0.327559 0.231638 30
produce:pandora:StandardPandora 3.03761 10.0632 39.3753 8.54612 6.68236 30
produce:pandoraWriter:StandardPandora 0.0537506 0.10417 0.226711 0.100519 0.0387017 30
produce:pandoraTrack:LArPandoraTrackCreation 0.27032 0.752831 2.57936 0.669367 0.43079 30
produce:pandoraShower:LArPandoraShowerCreation 0.365333 0.911638 2.05138 0.850899 0.446548 30
produce:pandoracalo:Calorimetry 0.178797 0.424356 1.22614 0.391012 0.197034 30
produce:pandoracalonosce:Calorimetry 0.149729 0.38123 1.16033 0.353338 0.184544 30
produce:pandorapid:Chi2ParticleID 0.000788304 0.00149258 0.00443603 0.00133472 0.000722051 30
produce:pandoraShowercalo:ShowerCalorimetry 0.320599 0.885087 1.89225 0.861286 0.400567 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.285143 0.850061 1.77819 0.834841 0.385992 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0574554 0.102376 0.199445 0.0893486 0.0412559 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0186121 0.0342049 0.0667605 0.0307546 0.0132067 30
produce:emtrkmichelid:EmTrackMichelId 11.9582 19.9068 33.9624 18.9393 5.20284 30
[art]:TriggerResults:TriggerResultInserter 1.5339e-05 2.17352e-05 5.5796e-05 1.82295e-05 8.11854e-06 30
end_path:out1:RootOutput 6.562e-06 8.39523e-06 1.8735e-05 7.3495e-06 2.52521e-06 30
end_path:michelremoving:michelremoving 0.0121669 0.0374059 0.106957 0.0206306 0.0307913 30
end_path:michelremovinggnocchi:michelremoving 0.00866117 0.0365857 0.123413 0.0173497 0.0350563 30
end_path:calibanagnocchi:CalibAnaTree 0.0998748 0.308315 0.816915 0.283167 0.138364 30
end_path:out1:RootOutput(write) 0.367038 0.613986 1.04594 0.620439 0.159779 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4940.45 MB
Peak resident set size usage (VmHWM): 2606.32 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0072_dataflow1_datawriter_0_20240906T114904_reco_stage1_reco_stage2_20240926T143520_keepup.root
\tHists: np04hd_raw_run029097_0072_dataflow1_datawriter_0_20240906T114904_reco_stage1_reco_stage2_20240926T143520_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
