Jobsub ID 263293.96@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263293.96@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3450 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:58:26 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:08:44 |
| From worker node | Hostname | ccwcondor0574 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:02:18 |
| Input files | hd-protodune:np04hd_raw_run029107_1519_dataflow0_datawriter_0_20240907T165457.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:08:44 |
| Saved logs | justin-logs:263293.96-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
C
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 364816 at 26-Sep-2024 12:56:44 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 54803 traces, 54803 summary
Retagger: tagging trace set: gauss with 40466 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40466 traces tagged "gauss"
FrameSaver: q=854532 n=733590 tag=gauss
wclsFrameSaver: saving 54803 traces tagged "wiener"
FrameSaver: q=922757 n=659246 tag=wiener
1600 hits in TPC 0
1389 hits in TPC 1
8487 hits in TPC 2
3705 hits in TPC 3
2208 hits in TPC 4
3639 hits in TPC 5
4717 hits in TPC 6
2932 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1343 XUs and 1375 XVs -> 59 XUVs
C:0 T:2 55546 XUs and 56745 XVs -> 1841 XUVs
C:0 T:3 157 XUs and 103 XVs -> 2 XUVs
C:0 T:4 13 XUs and 23 XVs -> 0 XUVs
C:0 T:5 6161 XUs and 7261 XVs -> 318 XUVs
C:0 T:6 54446 XUs and 69354 XVs -> 3391 XUVs
C:0 T:7 850 XUs and 1546 XVs -> 61 XUVs
5672 XUVs total
1485 collection wire objects
5672 potential space points
Neighbour search...
538866 tests to find 259632 neighbours
Iterating with no regularization...
Begin: 6.38779e+06
0 5.6177e+06
1 5.54798e+06
2 5.53468e+06
3 5.53016e+06
Now with regularization...
Begin: 5.2483e+06
0 5.2458e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 4 event 364816 TrkackID 0 startx, y, z -76.1367 503.517 454.877 wireno, peakt tpcno 10 2657.9 2 dist 0min T, max_T2657.74 4329.34
no of hits closeby 0 event 364816 TrkackID 9 startx, y, z 19.8441 593.317 116.946 wireno, peakt tpcno 237 3283.99 2 dist 328.349min T, max_T1055.19 3385.14
no of hits closeby 4 event 364816 TrkackID 0 startx, y, z -76.1367 503.517 454.877 wireno, peakt tpcno 10 2657.9 2 dist 0min T, max_T2657.74 4329.34
no of hits closeby 0 event 364816 TrkackID 9 startx, y, z 19.8441 593.317 116.946 wireno, peakt tpcno 237 3283.99 2 dist 328.349min T, max_T1055.19 3385.14
Begin processing the 30th record. run: 29107 subRun: 1 event: 364824 at 26-Sep-2024 12:57:48 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 50786 traces, 50786 summary
Retagger: tagging trace set: gauss with 36566 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 36566 traces tagged "gauss"
FrameSaver: q=848934 n=708699 tag=gauss
wclsFrameSaver: saving 50786 traces tagged "wiener"
FrameSaver: q=908869 n=638107 tag=wiener
2059 hits in TPC 0
2544 hits in TPC 1
7896 hits in TPC 2
4803 hits in TPC 3
2108 hits in TPC 4
2683 hits in TPC 5
2512 hits in TPC 6
2279 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 28 XUs and 50 XVs -> 0 XUVs
C:0 T:2 25250 XUs and 42459 XVs -> 2024 XUVs
C:0 T:3 161 XUs and 336 XVs -> 11 XUVs
C:0 T:4 102 XUs and 135 XVs -> 16 XUVs
C:0 T:5 3888 XUs and 5593 XVs -> 151 XUVs
C:0 T:6 5894 XUs and 6407 XVs -> 442 XUVs
C:0 T:7 83 XUs and 183 XVs -> 14 XUVs
2658 XUVs total
1398 collection wire objects
2658 potential space points
Neighbour search...
79468 tests to find 41260 neighbours
Iterating with no regularization...
Begin: 1.02846e+07
0 8.72286e+06
1 8.59603e+06
2 8.58412e+06
3 8.58244e+06
Now with regularization...
Begin: 7.99407e+06
0 7.99293e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 14 event 364824 TrkackID 12 startx, y, z -33.0319 393.961 377.966 wireno, peakt tpcno 309 203.639 6 dist 16.1831min T, max_T3.63031 416.121
no of hits closeby 14 event 364824 TrkackID 12 startx, y, z -33.0319 393.961 377.966 wireno, peakt tpcno 309 203.639 6 dist 16.1831min T, max_T3.63031 416.121
26-Sep-2024 12:58:58 CEST Closed output file "np04hd_raw_run029107_1519_dataflow0_datawriter_0_20240907T165457_reco_stage1_reco_stage2_20240926T105858_keepup.root"
26-Sep-2024 12:58:58 CEST Closed input file "np04hd_raw_run029107_1519_dataflow0_datawriter_0_20240907T165457_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 47.7869 80.8212 115.398 78.5305 14.7789 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000663824 0.00261018 0.00488322 0.00267597 0.00110235 30
produce:fembfilter:PDHDFEMBFilter 0.493248 0.545151 0.753196 0.510031 0.0736246 30
produce:wclsdatahd:WireCellToolkit 30.1158 33.1766 38.2795 33.3287 2.08141 30
produce:gaushit:GausHitFinder 0.278428 0.55863 0.968733 0.526608 0.140668 30
produce:nhitsfilter:NumberOfHitsFilter 0.000180037 0.000300732 0.000540073 0.000295043 6.18714e-05 30
produce:reco3d:SpacePointSolver 0.210901 0.805523 4.49556 0.611071 0.789394 30
produce:hitpdune:DisambigFromSpacePoints 0.09517 0.380355 0.772664 0.317464 0.159532 30
produce:pandora:StandardPandora 2.35039 10.2093 24.4521 8.28773 5.1304 30
produce:pandoraWriter:StandardPandora 0.0610444 0.135844 0.234882 0.129497 0.0384069 30
produce:pandoraTrack:LArPandoraTrackCreation 0.346367 1.54285 3.886 1.26724 0.745848 30
produce:pandoraShower:LArPandoraShowerCreation 0.272566 1.85861 4.26728 1.57119 0.944642 30
produce:pandoracalo:Calorimetry 0.184679 0.747798 1.61901 0.684749 0.299893 30
produce:pandoracalonosce:Calorimetry 0.154618 0.664178 1.3693 0.588427 0.271126 30
produce:pandorapid:Chi2ParticleID 0.00103295 0.00176667 0.00591786 0.00159932 0.000828183 30
produce:pandoraShowercalo:ShowerCalorimetry 0.222838 1.53736 2.90218 1.46535 0.605567 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.214898 1.44915 2.55871 1.41149 0.536334 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0476585 0.102977 0.207102 0.101218 0.0263349 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0184192 0.0391409 0.0675034 0.037103 0.0103687 30
produce:emtrkmichelid:EmTrackMichelId 11.8544 25.6284 37.8251 25.8614 5.42442 30
[art]:TriggerResults:TriggerResultInserter 3.233e-05 5.53928e-05 0.000123601 4.8972e-05 1.94992e-05 30
end_path:out1:RootOutput 1.04e-05 1.75134e-05 4.1758e-05 1.5539e-05 7.11498e-06 30
end_path:michelremoving:michelremoving 0.0256222 0.0796934 0.21641 0.0528459 0.055491 30
end_path:michelremovinggnocchi:michelremoving 0.0198734 0.0732412 0.20869 0.0449547 0.0576023 30
end_path:calibanagnocchi:CalibAnaTree 0.194168 0.492924 0.783125 0.471641 0.142245 30
end_path:out1:RootOutput(write) 0.356757 0.583707 0.845961 0.585271 0.114211 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4942.88 MB
Peak resident set size usage (VmHWM): 2362.86 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_1519_dataflow0_datawriter_0_20240907T165457_reco_stage1_reco_stage2_20240926T105858_keepup.root
\tHists: np04hd_raw_run029107_1519_dataflow0_datawriter_0_20240907T165457_reco_stage1_reco_stage2_20240926T105858_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
