Jobsub ID 263295.138@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263295.138@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3453 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:58:35 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:13:38 |
| From worker node | Hostname | ccwcondor0789 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:04:14 |
| Input files | hd-protodune:np04hd_raw_run029107_3440_dataflow5_datawriter_0_20240909T010929.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:13:38 |
| Saved logs | justin-logs:263295.138-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
t 2.47731min T, max_T4.48735 1946.4
no of hits closeby 9 event 825845 TrkackID 2 startx, y, z 151.327 555.59 202.574 wireno, peakt tpcno 455 796.767 5 dist 2.47731min T, max_T4.48735 1946.4
Begin processing the 29th record. run: 29107 subRun: 1 event: 825853 at 26-Sep-2024 12:58:22 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 82340 traces, 82340 summary
Retagger: tagging trace set: gauss with 60359 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 60359 traces tagged "gauss"
FrameSaver: q=1.41209e+06 n=1074912 tag=gauss
wclsFrameSaver: saving 82340 traces tagged "wiener"
FrameSaver: q=1.56342e+06 n=976750 tag=wiener
4298 hits in TPC 0
3068 hits in TPC 1
5350 hits in TPC 2
3505 hits in TPC 3
3806 hits in TPC 4
5912 hits in TPC 5
9566 hits in TPC 6
7885 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 463 XUs and 338 XVs -> 7 XUVs
C:0 T:2 25907 XUs and 14625 XVs -> 898 XUVs
C:0 T:3 503 XUs and 480 XVs -> 4 XUVs
C:0 T:4 361 XUs and 711 XVs -> 31 XUVs
C:0 T:5 22897 XUs and 33060 XVs -> 1391 XUVs
C:0 T:6 84268 XUs and 56268 XVs -> 2257 XUVs
C:0 T:7 4005 XUs and 3844 XVs -> 95 XUVs
4683 XUVs total
1693 collection wire objects
4683 potential space points
Neighbour search...
197269 tests to find 109942 neighbours
Iterating with no regularization...
Begin: 1.99399e+07
0 1.80877e+07
1 1.79365e+07
2 1.79175e+07
3 1.79128e+07
Now with regularization...
Begin: 1.70259e+07
0 1.70218e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 825853 TrkackID 13 startx, y, z 57.169 387.172 245.315 wireno, peakt tpcno 92 314.909 5 dist 0.0852535min T, max_T5.02597 727.137
no of hits closeby 0 event 825853 TrkackID 13 startx, y, z 57.169 387.172 245.315 wireno, peakt tpcno 92 314.909 5 dist 0.0852535min T, max_T5.02597 727.137
Begin processing the 30th record. run: 29107 subRun: 1 event: 825861 at 26-Sep-2024 12:59:55 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 53044 traces, 53044 summary
Retagger: tagging trace set: gauss with 40134 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 40134 traces tagged "gauss"
FrameSaver: q=838342 n=743315 tag=gauss
wclsFrameSaver: saving 53044 traces tagged "wiener"
FrameSaver: q=897279 n=676462 tag=wiener
3460 hits in TPC 0
3035 hits in TPC 1
2811 hits in TPC 2
1765 hits in TPC 3
3625 hits in TPC 4
6329 hits in TPC 5
3705 hits in TPC 6
2722 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 119 XUs and 45 XVs -> 0 XUVs
C:0 T:2 3385 XUs and 5620 XVs -> 719 XUVs
C:0 T:3 21 XUs and 32 XVs -> 1 XUVs
C:0 T:4 113 XUs and 223 XVs -> 9 XUVs
C:0 T:5 34284 XUs and 31377 XVs -> 1060 XUVs
C:0 T:6 5098 XUs and 6358 XVs -> 313 XUVs
C:0 T:7 384 XUs and 385 XVs -> 22 XUVs
2124 XUVs total
921 collection wire objects
2124 potential space points
Neighbour search...
84754 tests to find 53474 neighbours
Iterating with no regularization...
Begin: 4.06177e+06
0 3.85233e+06
1 3.83582e+06
2 3.83283e+06
Now with regularization...
Begin: 3.4995e+06
0 3.49887e+06
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000176013 0.000390172 0.000449657 0.000412703 0.000325441 0.000325441 0.000330448 0.000506163 0.000505924 0.000506163 0.000279307
----view 1: 9.73344e-05 9.73642e-05 0.00179613 0.00366735 0.00646019 0.00621986 0.00375748 0.00366783 0.00379276 0.00631905 0.00614738
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 3 event 825861 TrkackID 4 startx, y, z -101.191 585.341 461.143 wireno, peakt tpcno 169 2142.28 6 dist 0min T, max_T1046 2333.8
no of hits closeby 3 event 825861 TrkackID 4 startx, y, z -101.191 585.341 461.143 wireno, peakt tpcno 169 2142.28 6 dist 0min T, max_T1046 2333.8
26-Sep-2024 13:01:34 CEST Closed output file "np04hd_raw_run029107_3440_dataflow5_datawriter_0_20240909T010929_reco_stage1_reco_stage2_20240926T110134_keepup.root"
26-Sep-2024 13:01:34 CEST Closed input file "np04hd_raw_run029107_3440_dataflow5_datawriter_0_20240909T010929_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 54.6378 82.8618 121.203 84.5213 17.1862 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00042276 0.00258071 0.0051208 0.00260796 0.000982424 30
produce:fembfilter:PDHDFEMBFilter 0.492519 0.537886 0.667356 0.507973 0.0507534 30
produce:wclsdatahd:WireCellToolkit 30.7188 33.2895 38.797 32.819 2.08398 30
produce:gaushit:GausHitFinder 0.339266 0.546196 0.868837 0.537675 0.144285 30
produce:nhitsfilter:NumberOfHitsFilter 0.000198531 0.000286942 0.000486749 0.000271888 5.66495e-05 30
produce:reco3d:SpacePointSolver 0.118699 0.71255 1.3757 0.595548 0.390721 30
produce:hitpdune:DisambigFromSpacePoints 0.142695 0.390851 1.0285 0.294643 0.223876 30
produce:pandora:StandardPandora 3.08765 10.8076 23.1056 10.0985 4.97027 30
produce:pandoraWriter:StandardPandora 0.0737064 0.128295 0.203773 0.13438 0.0339753 30
produce:pandoraTrack:LArPandoraTrackCreation 0.522185 1.38815 3.02051 1.33815 0.682032 30
produce:pandoraShower:LArPandoraShowerCreation 0.414867 1.76407 3.62226 1.78061 0.965131 30
produce:pandoracalo:Calorimetry 0.236517 0.669392 1.32317 0.66863 0.273155 30
produce:pandoracalonosce:Calorimetry 0.241987 0.614197 1.2779 0.567069 0.262823 30
produce:pandorapid:Chi2ParticleID 0.00117423 0.00170639 0.00293488 0.00155208 0.000502423 30
produce:pandoraShowercalo:ShowerCalorimetry 0.393659 1.47721 3.36814 1.46836 0.782809 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.335578 1.43539 3.7775 1.35241 0.76592 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0594434 0.102819 0.170217 0.104301 0.0283173 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0236661 0.0378887 0.0762631 0.0381903 0.01152 30
produce:emtrkmichelid:EmTrackMichelId 15.5654 27.4347 44.9949 28.2655 7.20774 30
[art]:TriggerResults:TriggerResultInserter 3.4274e-05 5.38656e-05 9.4466e-05 4.92715e-05 1.62564e-05 30
end_path:out1:RootOutput 8.366e-06 1.60447e-05 2.6269e-05 1.4833e-05 3.68694e-06 30
end_path:michelremoving:michelremoving 0.0196568 0.113725 0.330896 0.0982314 0.0821065 30
end_path:michelremovinggnocchi:michelremoving 0.0163363 0.107846 0.324919 0.0917316 0.0802763 30
end_path:calibanagnocchi:CalibAnaTree 0.180491 0.491999 1.08471 0.454292 0.227465 30
end_path:out1:RootOutput(write) 0.36768 0.61065 1.0064 0.588248 0.155059 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4971.24 MB
Peak resident set size usage (VmHWM): 2388.8 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_3440_dataflow5_datawriter_0_20240909T010929_reco_stage1_reco_stage2_20240926T110134_keepup.root
\tHists: np04hd_raw_run029107_3440_dataflow5_datawriter_0_20240909T010929_reco_stage1_reco_stage2_20240926T110134_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
