Jobsub ID 263938.1@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263938.1@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3460 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:24:03 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce02 |
| Last heartbeat | 2024-09-26 15:38:20 |
| From worker node | Hostname | wn2204251.tier2.hep.manchester.ac.uk |
| cpuinfo | AMD EPYC 7513 32-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:25:59 |
| Input files | hd-protodune:np04hd_raw_run029218_0091_dataflow2_datawriter_0_20240917T164803.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:38:20 |
| Saved logs | justin-logs:263938.1-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
o regularization...
Begin: 1.44302e+07
0 1.3244e+07
1 1.31399e+07
2 1.3118e+07
3 1.31111e+07
Now with regularization...
Begin: 1.24728e+07
0 1.24681e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 22834 TrkackID 4 startx, y, z 164.595 603.005 167.614 wireno, peakt tpcno 307 3252.31 2 dist 359.854min T, max_T6.69834 3284.76
no of hits closeby 0 event 22834 TrkackID 6 startx, y, z 50.438 596.406 94.023 wireno, peakt tpcno 217 3795.88 2 dist 545.24min T, max_T6.9695 3835.07
no of hits closeby 0 event 22834 TrkackID 11 startx, y, z 128.298 321.368 117.312 wireno, peakt tpcno 358 1620.95 2 dist 85.362min T, max_T4.08699 1643.21
no of hits closeby 0 event 22834 TrkackID 4 startx, y, z 164.595 603.005 167.614 wireno, peakt tpcno 307 3252.31 2 dist 359.854min T, max_T6.69834 3284.76
no of hits closeby 0 event 22834 TrkackID 6 startx, y, z 50.438 596.406 94.023 wireno, peakt tpcno 217 3795.88 2 dist 545.24min T, max_T6.9695 3835.07
no of hits closeby 0 event 22834 TrkackID 11 startx, y, z 128.298 321.368 117.312 wireno, peakt tpcno 358 1620.95 2 dist 85.362min T, max_T4.08699 1643.21
Begin processing the 30th record. run: 29218 subRun: 1 event: 22842 at 26-Sep-2024 16:34:36 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 52825 traces, 52825 summary
Retagger: tagging trace set: gauss with 39808 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 39808 traces tagged "gauss"
FrameSaver: q=811616 n=754126 tag=gauss
wclsFrameSaver: saving 52825 traces tagged "wiener"
FrameSaver: q=888715 n=686200 tag=wiener
4328 hits in TPC 0
3853 hits in TPC 1
3792 hits in TPC 2
2743 hits in TPC 3
2562 hits in TPC 4
5484 hits in TPC 5
3245 hits in TPC 6
2043 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 27 XUs and 21 XVs -> 0 XUVs
C:0 T:1 6882 XUs and 5517 XVs -> 542 XUVs
C:0 T:2 6226 XUs and 23165 XVs -> 666 XUVs
C:0 T:3 31 XUs and 36 XVs -> 1 XUVs
C:0 T:4 275 XUs and 254 XVs -> 9 XUVs
C:0 T:5 13123 XUs and 10226 XVs -> 267 XUVs
C:0 T:6 4295 XUs and 6554 XVs -> 144 XUVs
C:0 T:7 113 XUs and 131 XVs -> 8 XUVs
1637 XUVs total
779 collection wire objects
1637 potential space points
Neighbour search...
63835 tests to find 39642 neighbours
Iterating with no regularization...
Begin: 4.71065e+06
0 4.58199e+06
1 4.57657e+06
2 4.57613e+06
Now with regularization...
Begin: 4.36554e+06
0 4.36536e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 31st record. run: 29218 subRun: 1 event: 22850 at 26-Sep-2024 16:35:47 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 79663 traces, 79663 summary
Retagger: tagging trace set: gauss with 57126 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 57126 traces tagged "gauss"
FrameSaver: q=1.22153e+06 n=1003129 tag=gauss
wclsFrameSaver: saving 79663 traces tagged "wiener"
FrameSaver: q=1.3172e+06 n=889451 tag=wiener
3964 hits in TPC 0
2369 hits in TPC 1
8318 hits in TPC 2
5059 hits in TPC 3
4073 hits in TPC 4
4945 hits in TPC 5
6744 hits in TPC 6
4768 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 132 XUs and 107 XVs -> 0 XUVs
C:0 T:2 53080 XUs and 34068 XVs -> 1454 XUVs
C:0 T:3 298 XUs and 241 XVs -> 6 XUVs
C:0 T:4 553 XUs and 697 XVs -> 19 XUVs
C:0 T:5 14418 XUs and 17580 XVs -> 384 XUVs
C:0 T:6 51805 XUs and 25985 XVs -> 775 XUVs
C:0 T:7 504 XUs and 491 XVs -> 7 XUVs
2645 XUVs total
1401 collection wire objects
2645 potential space points
Neighbour search...
41891 tests to find 25816 neighbours
Iterating with no regularization...
Begin: 1.06653e+07
0 9.98603e+06
1 9.94506e+06
2 9.94072e+06
Now with regularization...
Begin: 9.54897e+06
0 9.54786e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:37:47 BST Closed output file "np04hd_raw_run029218_0091_dataflow2_datawriter_0_20240917T164803_reco_stage1_reco_stage2_20240926T153747_keepup.root"
26-Sep-2024 16:37:47 BST Closed input file "np04hd_raw_run029218_0091_dataflow2_datawriter_0_20240917T164803_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 68.0083 89.6139 124.059 86.0038 13.8701 31
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000461317 0.00257036 0.00575664 0.00245535 0.00126832 31
produce:fembfilter:PDHDFEMBFilter 0.516928 0.739664 0.842127 0.743196 0.0507511 31
produce:wclsdatahd:WireCellToolkit 31.1103 38.3511 42.664 39.6504 3.04273 31
produce:gaushit:GausHitFinder 0.416751 0.755008 1.27216 0.757119 0.189134 31
produce:nhitsfilter:NumberOfHitsFilter 0.000244259 0.000361681 0.000670109 0.000356289 8.34919e-05 31
produce:reco3d:SpacePointSolver 0.181514 0.638267 1.62248 0.478248 0.366048 31
produce:hitpdune:DisambigFromSpacePoints 0.181766 0.551759 1.09909 0.530794 0.21612 31
produce:pandora:StandardPandora 6.23274 12.1648 28.3681 10.6334 5.07086 31
produce:pandoraWriter:StandardPandora 0.118348 0.20162 0.304583 0.205098 0.0517834 31
produce:pandoraTrack:LArPandoraTrackCreation 0.42232 1.11748 1.99135 1.06764 0.416327 31
produce:pandoraShower:LArPandoraShowerCreation 0.420536 1.33988 2.84924 1.15088 0.657465 31
produce:pandoracalo:Calorimetry 0.292155 0.570664 1.08158 0.539478 0.175529 31
produce:pandoracalonosce:Calorimetry 0.240766 0.506357 1.0024 0.486685 0.165972 31
produce:pandorapid:Chi2ParticleID 0.000985622 0.00194591 0.00474944 0.00191574 0.000664959 31
produce:pandoraShowercalo:ShowerCalorimetry 0.37453 1.19912 2.518 1.12989 0.562731 31
produce:pandoraShowercalonosce:ShowerCalorimetry 0.390602 1.17081 2.55184 1.07588 0.553358 31
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.081987 0.143276 0.227251 0.133481 0.0436722 31
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0261858 0.050625 0.0856237 0.0516773 0.0143066 31
produce:emtrkmichelid:EmTrackMichelId 18.5884 28.3209 42.04 27.1807 5.57639 31
[art]:TriggerResults:TriggerResultInserter 2.2011e-05 4.88747e-05 0.000105508 4.6327e-05 1.94747e-05 31
end_path:out1:RootOutput 6.823e-06 1.07891e-05 2.7491e-05 9.558e-06 3.77249e-06 31
end_path:michelremoving:michelremoving 0.0236095 0.112032 0.402136 0.0847681 0.0934587 31
end_path:michelremovinggnocchi:michelremoving 0.0204799 0.112481 0.407441 0.0826925 0.0970854 31
end_path:calibanagnocchi:CalibAnaTree 0.196191 0.465396 0.757258 0.478082 0.151892 31
end_path:out1:RootOutput(write) 0.595878 0.959263 1.40911 0.95328 0.203383 31
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4926.81 MB
Peak resident set size usage (VmHWM): 2570.22 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029218_0091_dataflow2_datawriter_0_20240917T164803_reco_stage1_reco_stage2_20240926T153747_keepup.root
\tHists: np04hd_raw_run029218_0091_dataflow2_datawriter_0_20240917T164803_reco_stage1_reco_stage2_20240926T153747_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
