Jobsub ID 263681.22@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263681.22@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:07:21 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:35:23 |
| From worker node | Hostname | n254.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:08:45 |
| Input files | hd-protodune:np04hd_raw_run029107_2251_dataflow2_datawriter_0_20240908T042139.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:35:23 |
| Saved logs | justin-logs:263681.22-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
2 XUs and 49 XVs -> 5 XUVs
C:0 T:5 33568 XUs and 36702 XVs -> 2225 XUVs
C:0 T:6 43606 XUs and 25465 XVs -> 1142 XUVs
C:0 T:7 71 XUs and 62 XVs -> 3 XUVs
4283 XUVs total
1512 collection wire objects
4283 potential space points
Neighbour search...
392583 tests to find 170830 neighbours
Iterating with no regularization...
Begin: 6.77816e+06
0 6.45005e+06
1 6.43122e+06
2 6.42807e+06
Now with regularization...
Begin: 6.07086e+06
0 6.06935e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 540490 at 26-Sep-2024 15:28:53 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 68859 traces, 68859 summary
Retagger: tagging trace set: gauss with 49993 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49993 traces tagged "gauss"
FrameSaver: q=1.07089e+06 n=897202 tag=gauss
wclsFrameSaver: saving 68859 traces tagged "wiener"
FrameSaver: q=1.17884e+06 n=810352 tag=wiener
1995 hits in TPC 0
3195 hits in TPC 1
10900 hits in TPC 2
6460 hits in TPC 3
1706 hits in TPC 4
2932 hits in TPC 5
5075 hits in TPC 6
3319 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 174 XUs and 446 XVs -> 6 XUVs
C:0 T:2 85871 XUs and 109342 XVs -> 2449 XUVs
C:0 T:3 338 XUs and 456 XVs -> 11 XUVs
C:0 T:4 0 XUs and 0 XVs -> 0 XUVs
C:0 T:5 19872 XUs and 9006 XVs -> 314 XUVs
C:0 T:6 13029 XUs and 17091 XVs -> 366 XUVs
C:0 T:7 97 XUs and 150 XVs -> 5 XUVs
3151 XUVs total
1626 collection wire objects
3151 potential space points
Neighbour search...
97143 tests to find 56492 neighbours
Iterating with no regularization...
Begin: 1.24107e+07
0 1.16346e+07
1 1.16021e+07
2 1.15988e+07
Now with regularization...
Begin: 1.10863e+07
0 1.10851e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 540498 at 26-Sep-2024 15:29:59 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 53201 traces, 53201 summary
Retagger: tagging trace set: gauss with 41007 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41007 traces tagged "gauss"
FrameSaver: q=923353 n=778484 tag=gauss
wclsFrameSaver: saving 53201 traces tagged "wiener"
FrameSaver: q=992523 n=711034 tag=wiener
2023 hits in TPC 0
2304 hits in TPC 1
3600 hits in TPC 2
2556 hits in TPC 3
1554 hits in TPC 4
3319 hits in TPC 5
8822 hits in TPC 6
5131 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2624 XUs and 1199 XVs -> 27 XUVs
C:0 T:2 9940 XUs and 7537 XVs -> 529 XUVs
C:0 T:3 210 XUs and 84 XVs -> 1 XUVs
C:0 T:4 1167 XUs and 3090 XVs -> 236 XUVs
C:0 T:5 4011 XUs and 4198 XVs -> 239 XUVs
C:0 T:6 35072 XUs and 31121 XVs -> 845 XUVs
C:0 T:7 618 XUs and 963 XVs -> 68 XUVs
1945 XUVs total
932 collection wire objects
1945 potential space points
Neighbour search...
75469 tests to find 41504 neighbours
Iterating with no regularization...
Begin: 6.15937e+06
0 5.62888e+06
1 5.57883e+06
2 5.56982e+06
3 5.56669e+06
Now with regularization...
Begin: 5.28326e+06
0 5.28191e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 540498 TrkackID 5 startx, y, z -155.92 425.434 232.201 wireno, peakt tpcno 138 5 6 dist 0min T, max_T5 2144.94
no of hits closeby 0 event 540498 TrkackID 8 startx, y, z 182.033 479.851 166.365 wireno, peakt tpcno 48 2292.34 2 dist 25.538min T, max_T2.69514 2310.9
no of hits closeby 0 event 540498 TrkackID 5 startx, y, z -155.92 425.434 232.201 wireno, peakt tpcno 138 5 6 dist 0min T, max_T5 2144.94
no of hits closeby 0 event 540498 TrkackID 8 startx, y, z 182.033 479.851 166.365 wireno, peakt tpcno 48 2292.34 2 dist 25.538min T, max_T2.69514 2310.9
26-Sep-2024 15:30:56 BST Closed output file "np04hd_raw_run029107_2251_dataflow2_datawriter_0_20240908T042139_reco_stage1_reco_stage2_20240926T143056_keepup.root"
26-Sep-2024 15:30:56 BST Closed input file "np04hd_raw_run029107_2251_dataflow2_datawriter_0_20240908T042139_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 46.0942 71.5431 113.288 67.5024 14.8298 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000406224 0.00222358 0.00372791 0.00227031 0.000802702 30
produce:fembfilter:PDHDFEMBFilter 0.451364 0.581761 0.712257 0.600246 0.116874 30
produce:wclsdatahd:WireCellToolkit 24.9387 30.6987 37.0525 31.0016 4.276 30
produce:gaushit:GausHitFinder 0.310003 0.584364 1.09564 0.559601 0.190597 30
produce:nhitsfilter:NumberOfHitsFilter 0.000217399 0.000340447 0.00103684 0.000300175 0.000148248 30
produce:reco3d:SpacePointSolver 0.157616 0.714302 2.04608 0.56452 0.542764 30
produce:hitpdune:DisambigFromSpacePoints 0.131855 0.415399 1.04456 0.365113 0.224842 30
produce:pandora:StandardPandora 3.68687 10.6495 42.4084 9.39887 7.09726 30
produce:pandoraWriter:StandardPandora 0.0927811 0.140127 0.214123 0.138775 0.0366874 30
produce:pandoraTrack:LArPandoraTrackCreation 0.391155 0.980922 1.98949 1.03708 0.363026 30
produce:pandoraShower:LArPandoraShowerCreation 0.309008 1.12011 3.08872 1.03978 0.573065 30
produce:pandoracalo:Calorimetry 0.221405 0.531522 1.04852 0.569279 0.185098 30
produce:pandoracalonosce:Calorimetry 0.189031 0.473794 0.921629 0.511737 0.168097 30
produce:pandorapid:Chi2ParticleID 0.000979933 0.00178874 0.00346567 0.00178937 0.000508913 30
produce:pandoraShowercalo:ShowerCalorimetry 0.306727 1.04777 2.66712 0.953038 0.523391 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.299256 1.01429 2.46121 0.903984 0.506712 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0676522 0.12674 0.236903 0.1332 0.0443527 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0237499 0.0435773 0.0754652 0.0445796 0.0147609 30
produce:emtrkmichelid:EmTrackMichelId 11.3982 21.1631 38.5393 21.065 5.32816 30
[art]:TriggerResults:TriggerResultInserter 2.149e-05 4.625e-05 0.000219233 3.6634e-05 3.58671e-05 30
end_path:out1:RootOutput 6.703e-06 1.16827e-05 2.7161e-05 1.02245e-05 4.94668e-06 30
end_path:michelremoving:michelremoving 0.0152131 0.0689758 0.252201 0.0273662 0.0769211 30
end_path:michelremovinggnocchi:michelremoving 0.0122285 0.0667469 0.253532 0.0241668 0.0774944 30
end_path:calibanagnocchi:CalibAnaTree 0.133397 0.342482 0.642084 0.286537 0.138252 30
end_path:out1:RootOutput(write) 0.39376 0.65524 1.16933 0.596712 0.206634 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4920.72 MB
Peak resident set size usage (VmHWM): 2598.56 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2251_dataflow2_datawriter_0_20240908T042139_reco_stage1_reco_stage2_20240926T143056_keepup.root
\tHists: np04hd_raw_run029107_2251_dataflow2_datawriter_0_20240908T042139_reco_stage1_reco_stage2_20240926T143056_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
