Jobsub ID 78782.116@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78782.116@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 12:35:26 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:06:21 |
| From worker node | Hostname | n260.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:37:32 |
| Input files | hd-protodune:np04hd_raw_run029072_0299_dataflow2_datawriter_0_20240903T205455.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:06:21 |
| Saved logs | justin-logs:78782.116-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
6 hits in TPC 2
2047 hits in TPC 3
3962 hits in TPC 4
6167 hits in TPC 5
9388 hits in TPC 6
9209 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1319 XUs and 1219 XVs -> 16 XUVs
C:0 T:2 7587 XUs and 12416 XVs -> 263 XUVs
C:0 T:3 118 XUs and 194 XVs -> 10 XUVs
C:0 T:4 194 XUs and 267 XVs -> 18 XUVs
C:0 T:5 13423 XUs and 15506 XVs -> 605 XUVs
C:0 T:6 66187 XUs and 55534 XVs -> 2611 XUVs
C:0 T:7 1582 XUs and 1718 XVs -> 58 XUVs
3581 XUVs total
1920 collection wire objects
3581 potential space points
Neighbour search...
270759 tests to find 85316 neighbours
Iterating with no regularization...
Begin: 1.51597e+07
0 1.32488e+07
1 1.31278e+07
2 1.31192e+07
Now with regularization...
Begin: 1.17863e+07
0 1.17789e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 71978 TrkackID 24 startx, y, z 4.36281 661.205 59.9644 wireno, peakt tpcno 200 1780.37 2 dist 0.312146min T, max_T1767 2374.07
no of hits closeby 0 event 71978 TrkackID 24 startx, y, z 4.36281 661.205 59.9644 wireno, peakt tpcno 200 1780.37 2 dist 0.312146min T, max_T1767 2374.07
Begin processing the 29th record. run: 29072 subRun: 1 event: 72026 at 26-Sep-2024 14:59:36 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66492 traces, 66492 summary
Retagger: tagging trace set: gauss with 48682 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48682 traces tagged "gauss"
FrameSaver: q=984375 n=886600 tag=gauss
wclsFrameSaver: saving 66492 traces tagged "wiener"
FrameSaver: q=1.0908e+06 n=811478 tag=wiener
3626 hits in TPC 0
3235 hits in TPC 1
7580 hits in TPC 2
3385 hits in TPC 3
3848 hits in TPC 4
5765 hits in TPC 5
5330 hits in TPC 6
2558 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3303 XUs and 2154 XVs -> 76 XUVs
C:0 T:2 51832 XUs and 90570 XVs -> 2250 XUVs
C:0 T:3 128 XUs and 238 XVs -> 9 XUVs
C:0 T:4 408 XUs and 472 XVs -> 19 XUVs
C:0 T:5 11233 XUs and 14028 XVs -> 406 XUVs
C:0 T:6 33191 XUs and 119897 XVs -> 1388 XUVs
C:0 T:7 74 XUs and 89 XVs -> 5 XUVs
4153 XUVs total
1423 collection wire objects
4153 potential space points
Neighbour search...
259469 tests to find 140242 neighbours
Iterating with no regularization...
Begin: 1.60515e+07
0 1.44133e+07
1 1.43106e+07
2 1.43041e+07
Now with regularization...
Begin: 1.40089e+07
0 1.4006e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29072 subRun: 1 event: 72034 at 26-Sep-2024 15:00:51 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 43460 traces, 43460 summary
Retagger: tagging trace set: gauss with 32166 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 32166 traces tagged "gauss"
FrameSaver: q=720731 n=579213 tag=gauss
wclsFrameSaver: saving 43460 traces tagged "wiener"
FrameSaver: q=773735 n=520221 tag=wiener
1917 hits in TPC 0
1978 hits in TPC 1
4404 hits in TPC 2
2515 hits in TPC 3
2727 hits in TPC 4
3904 hits in TPC 5
2692 hits in TPC 6
2197 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2177 XUs and 3967 XVs -> 108 XUVs
C:0 T:2 10210 XUs and 13371 XVs -> 683 XUVs
C:0 T:3 195 XUs and 256 XVs -> 18 XUVs
C:0 T:4 46 XUs and 97 XVs -> 6 XUVs
C:0 T:5 7086 XUs and 11468 XVs -> 191 XUVs
C:0 T:6 5443 XUs and 5927 XVs -> 525 XUVs
C:0 T:7 350 XUs and 350 XVs -> 6 XUVs
1537 XUVs total
873 collection wire objects
1537 potential space points
Neighbour search...
83755 tests to find 30620 neighbours
Iterating with no regularization...
Begin: 8.5795e+06
0 7.82151e+06
1 7.79486e+06
2 7.79313e+06
Now with regularization...
Begin: 7.1e+06
0 7.09869e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:01:50 BST Closed output file "np04hd_raw_run029072_0299_dataflow2_datawriter_0_20240903T205455_reco_stage1_reco_stage2_20240926T140150_keepup.root"
26-Sep-2024 15:01:50 BST Closed input file "np04hd_raw_run029072_0299_dataflow2_datawriter_0_20240903T205455_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 56.5967 79.8441 134.369 77.4249 16.5297 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000474407 0.00180442 0.00374981 0.00160675 0.000870503 30
produce:fembfilter:PDHDFEMBFilter 0.445943 0.605252 0.697543 0.629991 0.0917698 30
produce:wclsdatahd:WireCellToolkit 24.7572 32.3542 36.6299 33.5211 3.3037 30
produce:gaushit:GausHitFinder 0.37469 0.684292 1.14018 0.676969 0.177993 30
produce:nhitsfilter:NumberOfHitsFilter 0.000175882 0.000315372 0.00047046 0.000315672 6.18601e-05 30
produce:reco3d:SpacePointSolver 0.163292 0.720286 2.78639 0.521979 0.587673 30
produce:hitpdune:DisambigFromSpacePoints 0.147574 0.519167 1.3076 0.4707 0.259125 30
produce:pandora:StandardPandora 4.43174 12.16 42.4322 11.1491 7.5114 30
produce:pandoraWriter:StandardPandora 0.0865848 0.1477 0.290184 0.142406 0.0436735 30
produce:pandoraTrack:LArPandoraTrackCreation 0.42038 0.976718 2.52103 0.889977 0.448086 30
produce:pandoraShower:LArPandoraShowerCreation 0.434448 1.25244 4.62281 1.15822 0.739239 30
produce:pandoracalo:Calorimetry 0.281551 0.536017 1.34905 0.52352 0.211221 30
produce:pandoracalonosce:Calorimetry 0.244329 0.476473 1.23255 0.45017 0.196777 30
produce:pandorapid:Chi2ParticleID 0.00101005 0.00176675 0.00359943 0.00167995 0.000506368 30
produce:pandoraShowercalo:ShowerCalorimetry 0.397784 1.1392 3.82233 1.07622 0.604299 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.38248 1.0923 3.36843 1.03463 0.533277 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0719866 0.126322 0.174077 0.122945 0.0271407 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.024503 0.0428711 0.0591738 0.041267 0.00971887 30
produce:emtrkmichelid:EmTrackMichelId 16.4192 25.6109 42.3476 25.6073 5.32723 30
[art]:TriggerResults:TriggerResultInserter 1.9136e-05 3.33948e-05 0.000119155 2.74615e-05 2.06224e-05 30
end_path:out1:RootOutput 7.554e-06 1.09756e-05 2.7111e-05 1.0365e-05 3.41785e-06 30
end_path:michelremoving:michelremoving 0.0140444 0.0737206 0.282782 0.029907 0.0764371 30
end_path:michelremovinggnocchi:michelremoving 0.0134174 0.070895 0.270911 0.0289975 0.0752926 30
end_path:calibanagnocchi:CalibAnaTree 0.183716 0.395929 0.972114 0.368197 0.154551 30
end_path:out1:RootOutput(write) 0.467691 0.75496 1.30005 0.765846 0.182616 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4929.05 MB
Peak resident set size usage (VmHWM): 2596.44 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029072_0299_dataflow2_datawriter_0_20240903T205455_reco_stage1_reco_stage2_20240926T140150_keepup.root
\tHists: np04hd_raw_run029072_0299_dataflow2_datawriter_0_20240903T205455_reco_stage1_reco_stage2_20240926T140150_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
