Jobsub ID 263763.166@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263763.166@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:49 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 14:58:27 |
| From worker node | Hostname | wn093.hep |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:26:29 |
| Input files | hd-protodune:np04hd_raw_run029097_0086_dataflow7_datawriter_0_20240906T120542.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:58:27 |
| Saved logs | justin-logs:263763.166-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
1568 XVs -> 327 XUVs
C:0 T:7 24 XUs and 21 XVs -> 1 XUVs
2591 XUVs total
940 collection wire objects
2591 potential space points
Neighbour search...
244001 tests to find 108664 neighbours
Iterating with no regularization...
Begin: 4.76725e+06
0 4.36534e+06
1 4.33671e+06
2 4.33118e+06
3 4.32926e+06
Now with regularization...
Begin: 4.07487e+06
0 4.07349e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 20871 at 26-Sep-2024 15:54:18 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58209 traces, 58209 summary
Retagger: tagging trace set: gauss with 45973 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 45973 traces tagged "gauss"
FrameSaver: q=1.02109e+06 n=877630 tag=gauss
wclsFrameSaver: saving 58209 traces tagged "wiener"
FrameSaver: q=1.09272e+06 n=803654 tag=wiener
2415 hits in TPC 0
2756 hits in TPC 1
2818 hits in TPC 2
1287 hits in TPC 3
4451 hits in TPC 4
6542 hits in TPC 5
6562 hits in TPC 6
4906 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 395 XUs and 270 XVs -> 1 XUVs
C:0 T:2 46207 XUs and 8345 XVs -> 298 XUVs
C:0 T:3 55 XUs and 90 XVs -> 2 XUVs
C:0 T:4 169 XUs and 224 XVs -> 2 XUVs
C:0 T:5 57338 XUs and 34126 XVs -> 1153 XUVs
C:0 T:6 27510 XUs and 16147 XVs -> 675 XUVs
C:0 T:7 2105 XUs and 1127 XVs -> 71 XUVs
2202 XUVs total
1460 collection wire objects
2202 potential space points
Neighbour search...
29196 tests to find 17862 neighbours
Iterating with no regularization...
Begin: 1.3877e+07
0 1.31037e+07
1 1.30602e+07
2 1.30561e+07
Now with regularization...
Begin: 1.2647e+07
0 1.26455e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 20871 TrkackID 11 startx, y, z 27.7491 596.286 72.1236 wireno, peakt tpcno 173 3219.44 2 dist 277.043min T, max_T1262.2 3250.35
no of hits closeby 0 event 20871 TrkackID 11 startx, y, z 27.7491 596.286 72.1236 wireno, peakt tpcno 173 3219.44 2 dist 277.043min T, max_T1262.2 3250.35
Begin processing the 30th record. run: 29097 subRun: 1 event: 20879 at 26-Sep-2024 15:56:08 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 64158 traces, 64158 summary
Retagger: tagging trace set: gauss with 47018 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47018 traces tagged "gauss"
FrameSaver: q=1.04545e+06 n=898526 tag=gauss
wclsFrameSaver: saving 64158 traces tagged "wiener"
FrameSaver: q=1.13461e+06 n=816182 tag=wiener
4912 hits in TPC 0
3218 hits in TPC 1
7031 hits in TPC 2
3730 hits in TPC 3
2535 hits in TPC 4
4751 hits in TPC 5
5795 hits in TPC 6
3929 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 15 XUs and 2 XVs -> 0 XUVs
C:0 T:1 58 XUs and 44 XVs -> 1 XUVs
C:0 T:2 23549 XUs and 35564 XVs -> 1295 XUVs
C:0 T:3 314 XUs and 345 XVs -> 16 XUVs
C:0 T:4 229 XUs and 357 XVs -> 18 XUVs
C:0 T:5 31073 XUs and 45859 XVs -> 2295 XUVs
C:0 T:6 21916 XUs and 31952 XVs -> 3343 XUVs
C:0 T:7 132 XUs and 290 XVs -> 3 XUVs
6971 XUVs total
1613 collection wire objects
6971 potential space points
Neighbour search...
484855 tests to find 288060 neighbours
Iterating with no regularization...
Begin: 1.05313e+07
0 9.11981e+06
1 9.02201e+06
2 9.00755e+06
3 9.00192e+06
Now with regularization...
Begin: 8.21249e+06
0 8.20716e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:58:06 BST Closed output file "np04hd_raw_run029097_0086_dataflow7_datawriter_0_20240906T120542_reco_stage1_reco_stage2_20240926T145806_keepup.root"
26-Sep-2024 15:58:06 BST Closed input file "np04hd_raw_run029097_0086_dataflow7_datawriter_0_20240906T120542_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 90.7429 118.247 155.278 114.636 17.7331 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000769436 0.00293096 0.00485883 0.00326335 0.00117521 30
produce:fembfilter:PDHDFEMBFilter 0.693236 0.7492 0.809117 0.748875 0.0217304 30
produce:wclsdatahd:WireCellToolkit 52.8928 56.3056 58.5261 56.1888 1.26416 30
produce:gaushit:GausHitFinder 0.434431 0.791321 1.10532 0.775939 0.180551 30
produce:nhitsfilter:NumberOfHitsFilter 0.000298238 0.000425218 0.000530528 0.000445458 6.0406e-05 30
produce:reco3d:SpacePointSolver 0.307807 1.08134 6.01165 0.812262 1.10489 30
produce:hitpdune:DisambigFromSpacePoints 0.170809 0.574919 1.29424 0.528841 0.26609 30
produce:pandora:StandardPandora 6.33961 13.9616 25.2928 12.7457 4.98292 30
produce:pandoraWriter:StandardPandora 0.110404 0.172718 0.239854 0.167427 0.0333425 30
produce:pandoraTrack:LArPandoraTrackCreation 0.70748 2.13269 3.89604 1.98194 0.808176 30
produce:pandoraShower:LArPandoraShowerCreation 0.540462 2.67098 5.63156 2.54544 1.38298 30
produce:pandoracalo:Calorimetry 0.361113 0.94795 1.62913 0.894704 0.32346 30
produce:pandoracalonosce:Calorimetry 0.317286 0.878248 1.45798 0.834181 0.313617 30
produce:pandorapid:Chi2ParticleID 0.00149012 0.00208216 0.00289033 0.00207273 0.000315471 30
produce:pandoraShowercalo:ShowerCalorimetry 0.422669 2.25536 4.4828 2.14044 1.05986 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.419283 2.16586 4.48823 2.04188 1.00607 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0776604 0.143478 0.21243 0.14733 0.0328458 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0315237 0.0516812 0.0732709 0.0526923 0.0113739 30
produce:emtrkmichelid:EmTrackMichelId 18.8463 31.5442 45.6672 31.2868 6.5374 30
[art]:TriggerResults:TriggerResultInserter 2.2038e-05 4.54934e-05 0.000105988 4.12765e-05 1.61387e-05 30
end_path:out1:RootOutput 7.985e-06 1.27221e-05 3.667e-05 1.03175e-05 6.12942e-06 30
end_path:michelremoving:michelremoving 0.0277905 0.107945 0.341349 0.0565523 0.0966869 30
end_path:michelremovinggnocchi:michelremoving 0.0248458 0.100858 0.306334 0.0476596 0.0912765 30
end_path:calibanagnocchi:CalibAnaTree 0.25855 0.656383 1.03438 0.64071 0.219924 30
end_path:out1:RootOutput(write) 0.512903 0.804318 1.14197 0.77476 0.168178 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4930.25 MB
Peak resident set size usage (VmHWM): 2342.47 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0086_dataflow7_datawriter_0_20240906T120542_reco_stage1_reco_stage2_20240926T145806_keepup.root
\tHists: np04hd_raw_run029097_0086_dataflow7_datawriter_0_20240906T120542_reco_stage1_reco_stage2_20240926T145806_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
