Jobsub ID 263843.6@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263843.6@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:50:49 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 15:02:12 |
| From worker node | Hostname | wn026.hep |
| cpuinfo | Intel(R) Xeon(R) Gold 5215 CPU @ 2.50GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:51:59 |
| Input files | hd-protodune:np04hd_raw_run029094_0067_dataflow5_datawriter_0_20240906T090551.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:02:12 |
| Saved logs | justin-logs:263843.6-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
110 XUs and 108 XVs -> 4 XUVs
C:0 T:4 26 XUs and 32 XVs -> 2 XUVs
C:0 T:5 6553 XUs and 7333 XVs -> 329 XUVs
C:0 T:6 41142 XUs and 59125 XVs -> 3388 XUVs
C:0 T:7 1061 XUs and 931 XVs -> 21 XUVs
7471 XUVs total
2205 collection wire objects
7471 potential space points
Neighbour search...
622519 tests to find 292510 neighbours
Iterating with no regularization...
Begin: 1.71503e+07
0 1.36051e+07
1 1.33338e+07
2 1.33056e+07
3 1.32984e+07
Now with regularization...
Begin: 1.26358e+07
0 1.2632e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 16301 TrkackID 9 startx, y, z -110.506 401.168 53.2884 wireno, peakt tpcno 375 1125.31 2 dist 0.0205955min T, max_T12.4866 1657.76
no of hits closeby 2 event 16301 TrkackID 9 startx, y, z -110.506 401.168 53.2884 wireno, peakt tpcno 375 1125.31 2 dist 0.0205955min T, max_T12.4866 1657.76
Begin processing the 29th record. run: 29094 subRun: 1 event: 16309 at 26-Sep-2024 15:58:54 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 64963 traces, 64963 summary
Retagger: tagging trace set: gauss with 48612 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48612 traces tagged "gauss"
FrameSaver: q=1.06789e+06 n=902143 tag=gauss
wclsFrameSaver: saving 64963 traces tagged "wiener"
FrameSaver: q=1.13435e+06 n=817799 tag=wiener
3776 hits in TPC 0
3814 hits in TPC 1
5636 hits in TPC 2
3121 hits in TPC 3
3430 hits in TPC 4
4447 hits in TPC 5
6119 hits in TPC 6
2661 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 288 XUs and 284 XVs -> 5 XUVs
C:0 T:2 9067 XUs and 15565 XVs -> 1029 XUVs
C:0 T:3 142 XUs and 228 XVs -> 20 XUVs
C:0 T:4 814 XUs and 2805 XVs -> 95 XUVs
C:0 T:5 17099 XUs and 10262 XVs -> 471 XUVs
C:0 T:6 11438 XUs and 20228 XVs -> 491 XUVs
C:0 T:7 98 XUs and 156 XVs -> 7 XUVs
2118 XUVs total
1246 collection wire objects
2118 potential space points
Neighbour search...
34818 tests to find 21742 neighbours
Iterating with no regularization...
Begin: 7.89991e+06
0 7.53367e+06
1 7.51319e+06
2 7.51126e+06
Now with regularization...
Begin: 6.89345e+06
0 6.89292e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29094 subRun: 1 event: 16317 at 26-Sep-2024 16:00:25 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 49319 traces, 49319 summary
Retagger: tagging trace set: gauss with 37892 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 37892 traces tagged "gauss"
FrameSaver: q=832252 n=737458 tag=gauss
wclsFrameSaver: saving 49319 traces tagged "wiener"
FrameSaver: q=886909 n=668855 tag=wiener
3068 hits in TPC 0
3462 hits in TPC 1
3693 hits in TPC 2
1553 hits in TPC 3
2530 hits in TPC 4
4041 hits in TPC 5
4707 hits in TPC 6
4405 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 927 XUs and 240 XVs -> 7 XUVs
C:0 T:2 4671 XUs and 8045 XVs -> 360 XUVs
C:0 T:3 272 XUs and 365 XVs -> 17 XUVs
C:0 T:4 4 XUs and 1 XVs -> 0 XUVs
C:0 T:5 16171 XUs and 43656 XVs -> 1028 XUVs
C:0 T:6 15101 XUs and 24855 XVs -> 836 XUVs
C:0 T:7 604 XUs and 695 XVs -> 35 XUVs
2283 XUVs total
1057 collection wire objects
2283 potential space points
Neighbour search...
147157 tests to find 72368 neighbours
Iterating with no regularization...
Begin: 6.97117e+06
0 6.42319e+06
1 6.38418e+06
2 6.38114e+06
Now with regularization...
Begin: 6.04292e+06
0 6.04161e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:01:53 BST Closed output file "np04hd_raw_run029094_0067_dataflow5_datawriter_0_20240906T090551_reco_stage1_reco_stage2_20240926T150153_keepup.root"
26-Sep-2024 16:01:53 BST Closed input file "np04hd_raw_run029094_0067_dataflow5_datawriter_0_20240906T090551_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 63.4513 87.5304 116.795 88.2354 12.4482 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000342452 0.00127901 0.00330915 0.000983359 0.000879882 30
produce:fembfilter:PDHDFEMBFilter 0.622123 0.676754 0.824882 0.670451 0.0377361 30
produce:wclsdatahd:WireCellToolkit 39.9894 44.5827 51.4838 45.18 2.65723 30
produce:gaushit:GausHitFinder 0.338996 0.659608 1.01577 0.690054 0.167179 30
produce:nhitsfilter:NumberOfHitsFilter 0.000234911 0.000379 0.000584939 0.000366501 9.16014e-05 30
produce:reco3d:SpacePointSolver 0.092744 0.646477 2.60806 0.498151 0.496158 30
produce:hitpdune:DisambigFromSpacePoints 0.127329 0.45092 0.838054 0.463707 0.20627 30
produce:pandora:StandardPandora 4.30464 11.689 25.6366 11.4427 5.04792 30
produce:pandoraWriter:StandardPandora 0.0715082 0.148108 0.245451 0.146569 0.0373149 30
produce:pandoraTrack:LArPandoraTrackCreation 0.456826 1.42817 2.73569 1.37126 0.574014 30
produce:pandoraShower:LArPandoraShowerCreation 0.372074 1.66527 3.71575 1.43631 0.762303 30
produce:pandoracalo:Calorimetry 0.226381 0.645291 1.10712 0.650201 0.205974 30
produce:pandoracalonosce:Calorimetry 0.194516 0.593653 1.09631 0.594947 0.209114 30
produce:pandorapid:Chi2ParticleID 0.00112202 0.00335947 0.0533274 0.00160786 0.00928748 30
produce:pandoraShowercalo:ShowerCalorimetry 0.311144 1.53004 3.28995 1.33264 0.685717 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.305238 1.50837 3.46811 1.30474 0.726769 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0645775 0.110476 0.155597 0.111295 0.0244103 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0243072 0.0435355 0.0644764 0.0414317 0.0102227 30
produce:emtrkmichelid:EmTrackMichelId 10.1163 19.6953 32.6948 20.7402 4.60287 30
[art]:TriggerResults:TriggerResultInserter 2.5165e-05 3.71792e-05 9.2319e-05 3.4376e-05 1.21717e-05 30
end_path:out1:RootOutput 6.911e-06 1.04907e-05 2.5547e-05 9.3245e-06 3.5727e-06 30
end_path:michelremoving:michelremoving 0.0170466 0.0710304 0.308093 0.0403095 0.0618882 30
end_path:michelremovinggnocchi:michelremoving 0.014626 0.0675691 0.301183 0.0365578 0.0617953 30
end_path:calibanagnocchi:CalibAnaTree 0.133379 0.47122 0.928044 0.475738 0.169981 30
end_path:out1:RootOutput(write) 0.381055 0.671795 0.97881 0.681655 0.149769 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5840.06 MB
Peak resident set size usage (VmHWM): 2606.76 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029094_0067_dataflow5_datawriter_0_20240906T090551_reco_stage1_reco_stage2_20240926T150153_keepup.root
\tHists: np04hd_raw_run029094_0067_dataflow5_datawriter_0_20240906T090551_reco_stage1_reco_stage2_20240926T150153_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
