Jobsub ID 263692.8@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263692.8@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:12:17 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:32:39 |
| From worker node | Hostname | n260.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:14:06 |
| Input files | hd-protodune:np04hd_raw_run029107_1974_dataflow1_datawriter_0_20240907T235708.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:32:39 |
| Saved logs | justin-logs:263692.8-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
s
Neighbour search...
946700 tests to find 440516 neighbours
Iterating with no regularization...
Begin: 8.39711e+06
0 7.03058e+06
1 6.92043e+06
2 6.91305e+06
3 6.91071e+06
Now with regularization...
Begin: 6.44226e+06
0 6.441e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 474025 at 26-Sep-2024 15:29:01 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58691 traces, 58691 summary
Retagger: tagging trace set: gauss with 43021 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43021 traces tagged "gauss"
FrameSaver: q=873607 n=777518 tag=gauss
wclsFrameSaver: saving 58691 traces tagged "wiener"
FrameSaver: q=949189 n=698063 tag=wiener
3877 hits in TPC 0
3138 hits in TPC 1
6786 hits in TPC 2
2772 hits in TPC 3
1794 hits in TPC 4
4108 hits in TPC 5
4606 hits in TPC 6
2431 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 250 XUs and 961 XVs -> 43 XUVs
C:0 T:2 22075 XUs and 23586 XVs -> 2107 XUVs
C:0 T:3 64 XUs and 78 XVs -> 0 XUVs
C:0 T:4 13 XUs and 8 XVs -> 0 XUVs
C:0 T:5 20574 XUs and 54454 XVs -> 1213 XUVs
C:0 T:6 12610 XUs and 22508 XVs -> 640 XUVs
C:0 T:7 28 XUs and 15 XVs -> 0 XUVs
4003 XUVs total
990 collection wire objects
4003 potential space points
Neighbour search...
325395 tests to find 183270 neighbours
Iterating with no regularization...
Begin: 5.62295e+06
0 4.7051e+06
1 4.63346e+06
2 4.61975e+06
3 4.61474e+06
4 4.61194e+06
Now with regularization...
Begin: 4.25762e+06
0 4.25452e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 474033 at 26-Sep-2024 15:30:12 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69629 traces, 69629 summary
Retagger: tagging trace set: gauss with 53726 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 53726 traces tagged "gauss"
FrameSaver: q=1.14508e+06 n=976232 tag=gauss
wclsFrameSaver: saving 69629 traces tagged "wiener"
FrameSaver: q=1.22238e+06 n=880075 tag=wiener
2874 hits in TPC 0
2702 hits in TPC 1
7614 hits in TPC 2
3812 hits in TPC 3
4430 hits in TPC 4
5686 hits in TPC 5
6062 hits in TPC 6
3042 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 4812 XUs and 2825 XVs -> 72 XUVs
C:0 T:2 27512 XUs and 31402 XVs -> 619 XUVs
C:0 T:3 123 XUs and 164 XVs -> 4 XUVs
C:0 T:4 53 XUs and 72 XVs -> 2 XUVs
C:0 T:5 21995 XUs and 20544 XVs -> 450 XUVs
C:0 T:6 15117 XUs and 25506 XVs -> 1091 XUVs
C:0 T:7 11 XUs and 18 XVs -> 0 XUVs
2238 XUVs total
1439 collection wire objects
2238 potential space points
Neighbour search...
42764 tests to find 26474 neighbours
Iterating with no regularization...
Begin: 7.59508e+06
0 7.28891e+06
1 7.27755e+06
2 7.27652e+06
Now with regularization...
Begin: 6.77305e+06
0 6.77251e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 474033 TrkackID 0 startx, y, z 111.702 597.161 122.072 wireno, peakt tpcno 467 2144.12 5 dist 8.78061min T, max_T2066.97 5582.98
no of hits closeby 12 event 474033 TrkackID 6 startx, y, z 86.5334 593.466 417.456 wireno, peakt tpcno 475 2064.43 5 dist 5.38476min T, max_T2025.07 4265.88
no of hits closeby 0 event 474033 TrkackID 0 startx, y, z 111.702 597.161 122.072 wireno, peakt tpcno 467 2144.12 5 dist 8.78061min T, max_T2066.97 5582.98
no of hits closeby 12 event 474033 TrkackID 6 startx, y, z 86.5334 593.466 417.456 wireno, peakt tpcno 475 2064.43 5 dist 5.38476min T, max_T2025.07 4265.88
26-Sep-2024 15:31:26 BST Closed output file "np04hd_raw_run029107_1974_dataflow1_datawriter_0_20240907T235708_reco_stage1_reco_stage2_20240926T143126_keepup.root"
26-Sep-2024 15:31:26 BST Closed input file "np04hd_raw_run029107_1974_dataflow1_datawriter_0_20240907T235708_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 46.3902 67.1298 85.9349 67.9084 9.41165 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000316438 0.0017615 0.00404937 0.0019455 0.00084476 30
produce:fembfilter:PDHDFEMBFilter 0.44993 0.539852 0.700874 0.461152 0.0986466 30
produce:wclsdatahd:WireCellToolkit 24.4555 28.5055 35.5248 26.3389 3.81489 30
produce:gaushit:GausHitFinder 0.294934 0.542505 0.860336 0.513034 0.145966 30
produce:nhitsfilter:NumberOfHitsFilter 0.00020097 0.00027429 0.000484435 0.000263999 5.86847e-05 30
produce:reco3d:SpacePointSolver 0.169623 0.622047 2.1176 0.462823 0.50011 30
produce:hitpdune:DisambigFromSpacePoints 0.121489 0.371098 0.724612 0.361856 0.1516 30
produce:pandora:StandardPandora 3.46939 8.17492 14.2663 8.08095 2.59326 30
produce:pandoraWriter:StandardPandora 0.0627231 0.107855 0.163701 0.110911 0.0234326 30
produce:pandoraTrack:LArPandoraTrackCreation 0.486127 0.764426 1.28864 0.75919 0.209067 30
produce:pandoraShower:LArPandoraShowerCreation 0.416633 0.970639 1.91416 0.891143 0.365644 30
produce:pandoracalo:Calorimetry 0.283842 0.432054 0.660785 0.421248 0.10417 30
produce:pandoracalonosce:Calorimetry 0.254749 0.391168 0.608775 0.375784 0.100354 30
produce:pandorapid:Chi2ParticleID 0.000935819 0.00143301 0.00327962 0.00133202 0.000428804 30
produce:pandoraShowercalo:ShowerCalorimetry 0.371479 0.933694 1.77548 0.857408 0.34397 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.36756 0.90769 1.71811 0.838255 0.329122 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0651484 0.102937 0.178116 0.0997459 0.0271168 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0221841 0.0348763 0.0568975 0.0333998 0.00850082 30
produce:emtrkmichelid:EmTrackMichelId 13.9642 22.4987 35.9201 21.3491 5.13722 30
[art]:TriggerResults:TriggerResultInserter 2.0649e-05 2.69632e-05 7.1616e-05 2.4476e-05 9.889e-06 30
end_path:out1:RootOutput 9.518e-06 1.19731e-05 2.9346e-05 1.1066e-05 3.78892e-06 30
end_path:michelremoving:michelremoving 0.0127847 0.0677853 0.372357 0.0241248 0.0870021 30
end_path:michelremovinggnocchi:michelremoving 0.0105804 0.0641599 0.370263 0.0197072 0.0868782 30
end_path:calibanagnocchi:CalibAnaTree 0.175232 0.346513 0.903774 0.297745 0.158443 30
end_path:out1:RootOutput(write) 0.366661 0.655019 1.10349 0.660187 0.169469 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4942.14 MB
Peak resident set size usage (VmHWM): 2617.31 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_1974_dataflow1_datawriter_0_20240907T235708_reco_stage1_reco_stage2_20240926T143126_keepup.root
\tHists: np04hd_raw_run029107_1974_dataflow1_datawriter_0_20240907T235708_reco_stage1_reco_stage2_20240926T143126_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
