Jobsub ID 263757.94@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263757.94@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:23:33 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:55:48 |
| From worker node | Hostname | n253.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:25:33 |
| Input files | hd-protodune:np04hd_raw_run029094_0163_dataflow0_datawriter_0_20240906T095307.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:55:48 |
| Saved logs | justin-logs:263757.94-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
its in TPC 3
2374 hits in TPC 4
4295 hits in TPC 5
3334 hits in TPC 6
3073 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 77 XUs and 100 XVs -> 2 XUVs
C:0 T:2 15804 XUs and 6626 XVs -> 176 XUVs
C:0 T:3 26 XUs and 32 XVs -> 1 XUVs
C:0 T:4 238 XUs and 371 XVs -> 11 XUVs
C:0 T:5 5962 XUs and 7876 XVs -> 339 XUVs
C:0 T:6 6270 XUs and 8112 XVs -> 746 XUVs
C:0 T:7 158 XUs and 232 XVs -> 4 XUVs
1279 XUVs total
846 collection wire objects
1279 potential space points
Neighbour search...
33427 tests to find 15146 neighbours
Iterating with no regularization...
Begin: 4.66944e+06
0 4.39385e+06
1 4.37602e+06
2 4.37452e+06
Now with regularization...
Begin: 3.86931e+06
0 3.86803e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29094 subRun: 1 event: 39352 at 26-Sep-2024 15:52:05 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 56968 traces, 56968 summary
Retagger: tagging trace set: gauss with 43127 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43127 traces tagged "gauss"
FrameSaver: q=975214 n=812821 tag=gauss
wclsFrameSaver: saving 56968 traces tagged "wiener"
FrameSaver: q=1.04015e+06 n=731343 tag=wiener
3333 hits in TPC 0
3248 hits in TPC 1
5026 hits in TPC 2
3343 hits in TPC 3
3523 hits in TPC 4
4515 hits in TPC 5
4102 hits in TPC 6
3342 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 334 XUs and 256 XVs -> 17 XUVs
C:0 T:2 11436 XUs and 13994 XVs -> 285 XUVs
C:0 T:4 177 XUs and 204 XVs -> 4 XUVs
C:0 T:5 23639 XUs and 13352 XVs -> 538 XUVs
C:0 T:6 9629 XUs and 8808 XVs -> 384 XUVs
C:0 T:7 272 XUs and 242 XVs -> 19 XUVs
1247 XUVs total
882 collection wire objects
1247 potential space points
Neighbour search...
12543 tests to find 7030 neighbours
Iterating with no regularization...
Begin: 5.88966e+06
0 5.68142e+06
1 5.67501e+06
2 5.67463e+06
Now with regularization...
Begin: 5.32726e+06
0 5.3271e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 7 event 39352 TrkackID 10 startx, y, z -16.2331 312.695 14.2194 wireno, peakt tpcno 344 5260.27 2 dist 0min T, max_T5255.39 5855.03
no of hits closeby 7 event 39352 TrkackID 10 startx, y, z -16.2331 312.695 14.2194 wireno, peakt tpcno 344 5260.27 2 dist 0min T, max_T5255.39 5855.03
Begin processing the 30th record. run: 29094 subRun: 1 event: 39400 at 26-Sep-2024 15:53:23 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 63871 traces, 63871 summary
Retagger: tagging trace set: gauss with 48681 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48681 traces tagged "gauss"
FrameSaver: q=1.14086e+06 n=964360 tag=gauss
wclsFrameSaver: saving 63871 traces tagged "wiener"
FrameSaver: q=1.22019e+06 n=880816 tag=wiener
1793 hits in TPC 0
1716 hits in TPC 1
9339 hits in TPC 2
4104 hits in TPC 3
4901 hits in TPC 4
6603 hits in TPC 5
4551 hits in TPC 6
3299 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 25 XUs and 29 XVs -> 0 XUVs
C:0 T:2 31736 XUs and 38632 XVs -> 2043 XUVs
C:0 T:3 138 XUs and 213 XVs -> 3 XUVs
C:0 T:4 105 XUs and 95 XVs -> 3 XUVs
C:0 T:5 17582 XUs and 21265 XVs -> 794 XUVs
C:0 T:6 9969 XUs and 9734 XVs -> 278 XUVs
C:0 T:7 800 XUs and 643 XVs -> 23 XUVs
3144 XUVs total
1909 collection wire objects
3144 potential space points
Neighbour search...
135656 tests to find 55908 neighbours
Iterating with no regularization...
Begin: 1.23397e+07
0 1.07093e+07
1 1.06132e+07
2 1.0604e+07
Now with regularization...
Begin: 9.65597e+06
0 9.65059e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:55:10 BST Closed output file "np04hd_raw_run029094_0163_dataflow0_datawriter_0_20240906T095307_reco_stage1_reco_stage2_20240926T145510_keepup.root"
26-Sep-2024 15:55:10 BST Closed input file "np04hd_raw_run029094_0163_dataflow0_datawriter_0_20240906T095307_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 39.7661 83.3481 154.663 78.6618 21.0923 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000449592 0.00250701 0.00458177 0.00269418 0.000858889 30
produce:fembfilter:PDHDFEMBFilter 0.681464 0.700928 0.822221 0.696292 0.0241409 30
produce:wclsdatahd:WireCellToolkit 34.9508 36.2266 38.5242 36.123 0.812289 30
produce:gaushit:GausHitFinder 0.316595 0.736558 1.38754 0.697663 0.211098 30
produce:nhitsfilter:NumberOfHitsFilter 0.00023219 0.000358259 0.00073857 0.000336859 9.81353e-05 30
produce:reco3d:SpacePointSolver 0.099791 0.700872 2.97041 0.51052 0.585441 29
produce:hitpdune:DisambigFromSpacePoints 0.128159 0.587424 1.92042 0.506738 0.369326 29
produce:pandora:StandardPandora 2.84078 12.2825 39.4233 10.0281 7.0622 29
produce:pandoraWriter:StandardPandora 0.0690919 0.161966 0.281171 0.160247 0.0422004 29
produce:pandoraTrack:LArPandoraTrackCreation 0.237324 1.264 2.93259 1.16152 0.596762 29
produce:pandoraShower:LArPandoraShowerCreation 0.193043 1.45043 5.75584 1.15313 1.1131 29
produce:pandoracalo:Calorimetry 0.161859 0.674493 1.7677 0.617975 0.317203 29
produce:pandoracalonosce:Calorimetry 0.123369 0.60472 1.48793 0.560496 0.287545 29
produce:pandorapid:Chi2ParticleID 0.00136426 0.00408592 0.0588638 0.00198402 0.0103628 29
produce:pandoraShowercalo:ShowerCalorimetry 0.180663 1.36938 6.06264 1.04369 1.174 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.173848 1.2947 5.21372 0.999667 1.04561 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0655088 0.161253 0.322066 0.16058 0.0517821 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0232057 0.0537368 0.101624 0.0528848 0.0156585 29
produce:emtrkmichelid:EmTrackMichelId 11.947 25.0441 43.2811 24.0179 6.48857 29
[art]:TriggerResults:TriggerResultInserter 3.4606e-05 5.1337e-05 7.7347e-05 5.1613e-05 1.09604e-05 30
end_path:out1:RootOutput 8.325e-06 1.24218e-05 2.5268e-05 1.2063e-05 3.27003e-06 30
end_path:michelremoving:michelremoving 3.6719e-05 0.0828996 0.387253 0.036348 0.0888217 30
end_path:michelremovinggnocchi:michelremoving 2.167e-05 0.0794444 0.383383 0.0327281 0.0883396 30
end_path:calibanagnocchi:CalibAnaTree 1.9757e-05 0.452127 1.12251 0.416104 0.240768 30
end_path:out1:RootOutput(write) 0.354114 0.792692 1.67949 0.736459 0.240267 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4927.24 MB
Peak resident set size usage (VmHWM): 2599.4 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0163_dataflow0_datawriter_0_20240906T095307_reco_stage1_reco_stage2_20240926T145510_keepup.root
\tHists: np04hd_raw_run029094_0163_dataflow0_datawriter_0_20240906T095307_reco_stage1_reco_stage2_20240926T145510_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
