Jobsub ID 263527.1@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263527.1@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3461 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 11:18:55 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce3 |
| Last heartbeat | 2024-09-26 13:20:28 |
| From worker node | Hostname | n248.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 11:51:57 |
| Input files | hd-protodune:np04hd_raw_run027298_0192_dataflow0_datawriter_0_20240619T145750.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 13:20:28 |
| Saved logs | justin-logs:263527.1-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
.0187255
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 27012 TrkackID 1 startx, y, z 90.52 459.397 7.14206 wireno, peakt tpcno 341 5415.81 2 dist 393.229min T, max_T3383.11 5446.71
no of hits closeby 0 event 27012 TrkackID 1 startx, y, z 90.52 459.397 7.14206 wireno, peakt tpcno 341 5415.81 2 dist 393.229min T, max_T3383.11 5446.71
Begin processing the 34th record. run: 27298 subRun: 1 event: 27016 at 26-Sep-2024 14:08:47 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 45636 traces, 45636 summary
Retagger: tagging trace set: gauss with 34927 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 34927 traces tagged "gauss"
FrameSaver: q=1.33157e+06 n=656258 tag=gauss
wclsFrameSaver: saving 45636 traces tagged "wiener"
FrameSaver: q=1.42126e+06 n=603960 tag=wiener
1020 hits in TPC 0
520 hits in TPC 1
4977 hits in TPC 2
1732 hits in TPC 3
3185 hits in TPC 4
5993 hits in TPC 5
5090 hits in TPC 6
3868 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 49 XUs and 42 XVs -> 0 XUVs
C:0 T:2 5492 XUs and 10171 XVs -> 450 XUVs
C:0 T:3 319 XUs and 415 XVs -> 8 XUVs
C:0 T:4 155 XUs and 137 XVs -> 4 XUVs
C:0 T:5 12560 XUs and 12272 XVs -> 504 XUVs
C:0 T:6 13753 XUs and 26455 XVs -> 772 XUVs
C:0 T:7 125 XUs and 242 XVs -> 3 XUVs
1741 XUVs total
1138 collection wire objects
1741 potential space points
Neighbour search...
25517 tests to find 14958 neighbours
Iterating with no regularization...
Begin: 2.02247e+07
0 1.90906e+07
1 1.90505e+07
2 1.90429e+07
Now with regularization...
Begin: 1.80201e+07
0 1.8018e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 27016 TrkackID 0 startx, y, z 278.606 590.72 66.1061 wireno, peakt tpcno 172 10.5 5 dist 0min T, max_T5 3529.66
no of hits closeby 2 event 27016 TrkackID 0 startx, y, z 278.606 590.72 66.1061 wireno, peakt tpcno 172 10.5 5 dist 0min T, max_T5 3529.66
Begin processing the 35th record. run: 27298 subRun: 1 event: 27020 at 26-Sep-2024 14:09:54 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65537 traces, 65537 summary
Retagger: tagging trace set: gauss with 50953 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50953 traces tagged "gauss"
FrameSaver: q=1.83502e+06 n=987430 tag=gauss
wclsFrameSaver: saving 65537 traces tagged "wiener"
FrameSaver: q=1.94747e+06 n=900539 tag=wiener
5216 hits in TPC 0
5244 hits in TPC 1
6927 hits in TPC 2
4163 hits in TPC 3
1357 hits in TPC 4
3390 hits in TPC 5
6393 hits in TPC 6
4721 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 4948 XUs and 7912 XVs -> 635 XUVs
C:0 T:2 13476 XUs and 18046 XVs -> 471 XUVs
C:0 T:3 612 XUs and 895 XVs -> 30 XUVs
C:0 T:4 38 XUs and 43 XVs -> 0 XUVs
C:0 T:5 4590 XUs and 6592 XVs -> 240 XUVs
C:0 T:6 17236 XUs and 18814 XVs -> 511 XUVs
C:0 T:7 552 XUs and 486 XVs -> 18 XUVs
1905 XUVs total
1097 collection wire objects
1905 potential space points
Neighbour search...
27051 tests to find 15922 neighbours
Iterating with no regularization...
Begin: 1.74972e+07
0 1.7074e+07
1 1.70513e+07
2 1.70498e+07
Now with regularization...
Begin: 1.62629e+07
0 1.62625e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 14:11:00 BST Closed output file "np04hd_raw_run027298_0192_dataflow0_datawriter_0_20240619T145750_reco_stage1_reco_stage2_20240926T131100_keepup.root"
26-Sep-2024 14:11:00 BST Closed input file "np04hd_raw_run027298_0192_dataflow0_datawriter_0_20240619T145750_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 45.0834 68.1707 107.788 67.0303 14.7692 35
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000339532 0.00140246 0.00326444 0.00147312 0.000720785 35
produce:fembfilter:PDHDFEMBFilter 0.487389 0.63922 0.768113 0.711866 0.1177 35
produce:wclsdatahd:WireCellToolkit 23.7399 30.004 35.9217 33.4606 5.04537 35
produce:gaushit:GausHitFinder 0.294895 0.614564 1.22884 0.583763 0.21934 35
produce:nhitsfilter:NumberOfHitsFilter 0.000174009 0.000273228 0.000481079 0.000259941 6.82711e-05 35
produce:reco3d:SpacePointSolver 0.112436 0.612981 2.53978 0.40695 0.497284 35
produce:hitpdune:DisambigFromSpacePoints 0.137471 0.471334 0.931681 0.4496 0.231588 35
produce:pandora:StandardPandora 3.48731 10.1236 23.5495 9.74879 4.57569 35
produce:pandoraWriter:StandardPandora 0.0530777 0.113943 0.212575 0.10487 0.0404149 35
produce:pandoraTrack:LArPandoraTrackCreation 0.437144 0.930621 2.0482 0.882509 0.357893 35
produce:pandoraShower:LArPandoraShowerCreation 0.434065 1.06598 2.74766 0.959865 0.510002 35
produce:pandoracalo:Calorimetry 0.242713 0.510528 1.04542 0.49047 0.187609 35
produce:pandoracalonosce:Calorimetry 0.220471 0.463517 0.986721 0.444623 0.169773 35
produce:pandorapid:Chi2ParticleID 0.000876446 0.00165319 0.00367538 0.00152406 0.000549665 35
produce:pandoraShowercalo:ShowerCalorimetry 0.376185 1.03465 2.55367 0.930555 0.469113 35
produce:pandoraShowercalonosce:ShowerCalorimetry 0.364582 0.997868 2.50572 0.892628 0.451529 35
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.049919 0.114204 0.229394 0.10938 0.0418174 35
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0171724 0.0375297 0.0769295 0.0358469 0.0137708 35
produce:emtrkmichelid:EmTrackMichelId 11.8723 19.1651 30.326 18.7765 4.42365 35
[art]:TriggerResults:TriggerResultInserter 1.4798e-05 2.51159e-05 8.2096e-05 2.5087e-05 1.09201e-05 35
end_path:out1:RootOutput 6.873e-06 1.02133e-05 2.7953e-05 9.578e-06 3.46025e-06 35
end_path:michelremoving:michelremoving 0.0112076 0.0562488 0.266012 0.0255469 0.0603808 35
end_path:michelremovinggnocchi:michelremoving 0.0103215 0.0539721 0.266191 0.0219733 0.0606952 35
end_path:calibanagnocchi:CalibAnaTree 0.151721 0.402092 0.90792 0.362489 0.143953 35
end_path:out1:RootOutput(write) 0.372743 0.681427 1.08048 0.671281 0.198241 35
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4949.98 MB
Peak resident set size usage (VmHWM): 2617.45 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027298_0192_dataflow0_datawriter_0_20240619T145750_reco_stage1_reco_stage2_20240926T131100_keepup.root
\tHists: np04hd_raw_run027298_0192_dataflow0_datawriter_0_20240619T145750_reco_stage1_reco_stage2_20240926T131100_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
