Jobsub ID 263236.0@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263236.0@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:30:33 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 10:47:40 |
| From worker node | Hostname | ccwcondor0483 |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:32:18 |
| Input files | hd-protodune:np04hd_raw_run029107_0740_dataflow4_datawriter_0_20240907T030914.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 10:47:40 |
| Saved logs | justin-logs:263236.0-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
ion...
Begin: 7.79209e+06
0 7.29984e+06
1 7.25321e+06
2 7.24252e+06
3 7.23997e+06
Now with regularization...
Begin: 6.83285e+06
0 6.83137e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 177852 at 26-Sep-2024 12:41:55 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 90029 traces, 90029 summary
Retagger: tagging trace set: gauss with 66213 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 66213 traces tagged "gauss"
FrameSaver: q=1.36765e+06 n=1247187 tag=gauss
wclsFrameSaver: saving 90029 traces tagged "wiener"
FrameSaver: q=1.47025e+06 n=1119943 tag=wiener
6259 hits in TPC 0
2913 hits in TPC 1
10245 hits in TPC 2
5442 hits in TPC 3
3282 hits in TPC 4
5211 hits in TPC 5
10566 hits in TPC 6
5450 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2814 XUs and 2047 XVs -> 66 XUVs
C:0 T:2 74889 XUs and 104196 XVs -> 1194 XUVs
C:0 T:3 114 XUs and 109 XVs -> 4 XUVs
C:0 T:4 477 XUs and 1044 XVs -> 35 XUVs
C:0 T:5 36269 XUs and 46000 XVs -> 474 XUVs
C:0 T:6 90918 XUs and 102526 XVs -> 2258 XUVs
C:0 T:7 1399 XUs and 1241 XVs -> 65 XUVs
4096 XUVs total
2055 collection wire objects
4096 potential space points
Neighbour search...
113060 tests to find 61662 neighbours
Iterating with no regularization...
Begin: 1.22113e+07
0 1.13229e+07
1 1.12704e+07
2 1.1263e+07
Now with regularization...
Begin: 1.0804e+07
0 1.08016e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 177860 at 26-Sep-2024 12:44:21 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 76008 traces, 76008 summary
Retagger: tagging trace set: gauss with 55684 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 55684 traces tagged "gauss"
FrameSaver: q=1.29211e+06 n=1050750 tag=gauss
wclsFrameSaver: saving 76008 traces tagged "wiener"
FrameSaver: q=1.40817e+06 n=953582 tag=wiener
5184 hits in TPC 0
3718 hits in TPC 1
7768 hits in TPC 2
3578 hits in TPC 3
4711 hits in TPC 4
7860 hits in TPC 5
5047 hits in TPC 6
3167 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 5115 XUs and 5388 XVs -> 80 XUVs
C:0 T:2 18095 XUs and 19707 XVs -> 580 XUVs
C:0 T:3 236 XUs and 238 XVs -> 12 XUVs
C:0 T:4 441 XUs and 467 XVs -> 21 XUVs
C:0 T:5 49932 XUs and 24033 XVs -> 2034 XUVs
C:0 T:6 12064 XUs and 23911 XVs -> 2163 XUVs
C:0 T:7 65 XUs and 157 XVs -> 5 XUVs
4895 XUVs total
1859 collection wire objects
4895 potential space points
Neighbour search...
351703 tests to find 151864 neighbours
Iterating with no regularization...
Begin: 2.02244e+07
0 1.69774e+07
1 1.67713e+07
2 1.6743e+07
3 1.67365e+07
Now with regularization...
Begin: 1.57516e+07
0 1.57453e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 177860 TrkackID 0 startx, y, z -222.653 465.17 322.901 wireno, peakt tpcno 5 608.948 2 dist 0min T, max_T4.83686 2865.78
no of hits closeby 0 event 177860 TrkackID 11 startx, y, z 143.669 602.468 41.4503 wireno, peakt tpcno 138 4999.5 2 dist 183.473min T, max_T3225.68 5026.28
no of hits closeby 0 event 177860 TrkackID 0 startx, y, z -222.653 465.17 322.901 wireno, peakt tpcno 5 608.948 2 dist 0min T, max_T4.83686 2865.78
no of hits closeby 0 event 177860 TrkackID 11 startx, y, z 143.669 602.468 41.4503 wireno, peakt tpcno 138 4999.5 2 dist 183.473min T, max_T3225.68 5026.28
26-Sep-2024 12:46:27 CEST Closed output file "np04hd_raw_run029107_0740_dataflow4_datawriter_0_20240907T030914_reco_stage1_reco_stage2_20240926T104627_keepup.root"
26-Sep-2024 12:46:27 CEST Closed input file "np04hd_raw_run029107_0740_dataflow4_datawriter_0_20240907T030914_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 76.8069 105.183 151.172 102.166 18.6896 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000632039 0.00297224 0.00678142 0.00310252 0.00134923 30
produce:fembfilter:PDHDFEMBFilter 0.587367 0.892148 2.28127 0.85171 0.286572 30
produce:wclsdatahd:WireCellToolkit 40.2658 45.3423 49.3247 45.8334 2.74131 30
produce:gaushit:GausHitFinder 0.403086 0.767123 1.19435 0.758435 0.190585 30
produce:nhitsfilter:NumberOfHitsFilter 0.000290577 0.000432922 0.000628282 0.000433606 7.58326e-05 30
produce:reco3d:SpacePointSolver 0.193517 1.19763 4.55799 0.788429 1.0139 30
produce:hitpdune:DisambigFromSpacePoints 0.192505 0.571474 1.17828 0.492045 0.274071 30
produce:pandora:StandardPandora 5.89886 14.8203 36.23 12.0084 7.60306 30
produce:pandoraWriter:StandardPandora 0.0834395 0.169684 0.235085 0.169889 0.0365189 30
produce:pandoraTrack:LArPandoraTrackCreation 0.59579 1.3239 3.1763 1.15284 0.544899 30
produce:pandoraShower:LArPandoraShowerCreation 0.550219 1.48599 3.2562 1.45355 0.675133 30
produce:pandoracalo:Calorimetry 0.349583 0.64111 1.39656 0.625252 0.214706 30
produce:pandoracalonosce:Calorimetry 0.330214 0.579516 1.5786 0.549225 0.235193 30
produce:pandorapid:Chi2ParticleID 0.00148513 0.00230528 0.00607082 0.00210338 0.000920042 30
produce:pandoraShowercalo:ShowerCalorimetry 0.368359 1.35525 2.82414 1.33571 0.584056 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.364639 1.30713 2.6938 1.292 0.549059 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.106927 0.163359 0.283863 0.165926 0.0388906 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.033671 0.0539817 0.077422 0.0537598 0.0121633 30
produce:emtrkmichelid:EmTrackMichelId 19.9748 32.8624 47.187 32.2516 6.80548 30
[art]:TriggerResults:TriggerResultInserter 2.3995e-05 5.22777e-05 0.000110959 4.238e-05 2.46554e-05 30
end_path:out1:RootOutput 9.538e-06 1.38923e-05 3.203e-05 1.1422e-05 6.11906e-06 30
end_path:michelremoving:michelremoving 0.0193449 0.0989918 0.339968 0.0564989 0.0874126 30
end_path:michelremovinggnocchi:michelremoving 0.0167916 0.0921405 0.331634 0.0440952 0.0842188 30
end_path:calibanagnocchi:CalibAnaTree 0.221583 0.468084 1.05027 0.453482 0.186712 30
end_path:out1:RootOutput(write) 0.515036 0.837987 1.23588 0.853009 0.16185 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4949.86 MB
Peak resident set size usage (VmHWM): 2369.16 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0740_dataflow4_datawriter_0_20240907T030914_reco_stage1_reco_stage2_20240926T104627_keepup.root
\tHists: np04hd_raw_run029107_0740_dataflow4_datawriter_0_20240907T030914_reco_stage1_reco_stage2_20240926T104627_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
