Jobsub ID 263681.59@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263681.59@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:07:21 |
| Site | UK_Durham |
| Entry | DUNE_UK_SGridDurham_ce4 |
| Last heartbeat | 2024-09-26 14:31:41 |
| From worker node | Hostname | n245.dur.scotgrid.ac.uk |
| cpuinfo | AMD EPYC 7763 64-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 171000 (47 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:08:28 |
| Input files | hd-protodune:np04hd_raw_run029107_2166_dataflow7_datawriter_0_20240908T024601.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:31:41 |
| Saved logs | justin-logs:263681.59-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
6 neighbours
Iterating with no regularization...
Begin: 4.21904e+06
0 3.79138e+06
1 3.76774e+06
2 3.7656e+06
Now with regularization...
Begin: 3.54033e+06
0 3.53946e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 520079 TrkackID 8 startx, y, z 86.8937 360.96 11.6428 wireno, peakt tpcno 87 1054.74 2 dist 116.415min T, max_T5 1086.5
no of hits closeby 2 event 520079 TrkackID 20 startx, y, z -26.0138 118.083 240.938 wireno, peakt tpcno 421 23.8269 2 dist 0.0124781min T, max_T5 320.227
no of hits closeby 0 event 520079 TrkackID 8 startx, y, z 86.8937 360.96 11.6428 wireno, peakt tpcno 87 1054.74 2 dist 116.415min T, max_T5 1086.5
no of hits closeby 2 event 520079 TrkackID 20 startx, y, z -26.0138 118.083 240.938 wireno, peakt tpcno 421 23.8269 2 dist 0.0124781min T, max_T5 320.227
Begin processing the 29th record. run: 29107 subRun: 1 event: 520087 at 26-Sep-2024 15:27:26 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66194 traces, 66194 summary
Retagger: tagging trace set: gauss with 49407 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49407 traces tagged "gauss"
FrameSaver: q=1.14017e+06 n=914523 tag=gauss
wclsFrameSaver: saving 66194 traces tagged "wiener"
FrameSaver: q=1.25676e+06 n=836091 tag=wiener
4128 hits in TPC 0
4539 hits in TPC 1
5375 hits in TPC 2
2537 hits in TPC 3
4823 hits in TPC 4
6852 hits in TPC 5
4763 hits in TPC 6
3074 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3618 XUs and 4169 XVs -> 108 XUVs
C:0 T:2 20168 XUs and 24534 XVs -> 1029 XUVs
C:0 T:3 25 XUs and 45 XVs -> 0 XUVs
C:0 T:4 99 XUs and 84 XVs -> 1 XUVs
C:0 T:5 46609 XUs and 81896 XVs -> 3792 XUVs
C:0 T:6 9254 XUs and 14222 XVs -> 453 XUVs
C:0 T:7 23 XUs and 56 XVs -> 1 XUVs
5384 XUVs total
1871 collection wire objects
5384 potential space points
Neighbour search...
394932 tests to find 126318 neighbours
Iterating with no regularization...
Begin: 2.30748e+07
0 1.5342e+07
1 1.47759e+07
2 1.46995e+07
3 1.46801e+07
4 1.46742e+07
Now with regularization...
Begin: 1.37814e+07
0 1.37729e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 520095 at 26-Sep-2024 15:28:44 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 63054 traces, 63054 summary
Retagger: tagging trace set: gauss with 46195 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 46195 traces tagged "gauss"
FrameSaver: q=985289 n=867595 tag=gauss
wclsFrameSaver: saving 63054 traces tagged "wiener"
FrameSaver: q=1.06196e+06 n=781394 tag=wiener
3876 hits in TPC 0
4087 hits in TPC 1
6088 hits in TPC 2
3026 hits in TPC 3
2811 hits in TPC 4
3866 hits in TPC 5
5836 hits in TPC 6
3090 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 560 XUs and 412 XVs -> 5 XUVs
C:0 T:2 15301 XUs and 42542 XVs -> 748 XUVs
C:0 T:3 95 XUs and 111 XVs -> 1 XUVs
C:0 T:4 84 XUs and 104 XVs -> 2 XUVs
C:0 T:5 9587 XUs and 13080 XVs -> 345 XUVs
C:0 T:6 23912 XUs and 73656 XVs -> 1669 XUVs
C:0 T:7 58 XUs and 101 XVs -> 4 XUVs
2774 XUVs total
1313 collection wire objects
2774 potential space points
Neighbour search...
113434 tests to find 64078 neighbours
Iterating with no regularization...
Begin: 1.1067e+07
0 1.03264e+07
1 1.02961e+07
2 1.02932e+07
Now with regularization...
Begin: 9.81656e+06
0 9.81515e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:29:59 BST Closed output file "np04hd_raw_run029107_2166_dataflow7_datawriter_0_20240908T024601_reco_stage1_reco_stage2_20240926T142959_keepup.root"
26-Sep-2024 15:29:59 BST Closed input file "np04hd_raw_run029107_2166_dataflow7_datawriter_0_20240908T024601_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 51.3349 73.1204 106.277 74.8471 11.7695 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000357684 0.00214979 0.00362666 0.00206389 0.000958081 30
produce:fembfilter:PDHDFEMBFilter 0.450378 0.610582 0.69589 0.660341 0.0933924 30
produce:wclsdatahd:WireCellToolkit 26.4388 33.2982 35.8839 34.2815 2.42459 30
produce:gaushit:GausHitFinder 0.228128 0.647774 0.963366 0.67065 0.171182 30
produce:nhitsfilter:NumberOfHitsFilter 0.000163328 0.000303812 0.00048833 0.000296158 7.08634e-05 30
produce:reco3d:SpacePointSolver 0.108837 0.653329 1.78795 0.532621 0.413106 30
produce:hitpdune:DisambigFromSpacePoints 0.111126 0.469604 1.02947 0.479293 0.230032 30
produce:pandora:StandardPandora 2.172 9.47635 17.4568 9.1809 3.3864 30
produce:pandoraWriter:StandardPandora 0.0546664 0.124759 0.236402 0.128227 0.03875 30
produce:pandoraTrack:LArPandoraTrackCreation 0.278603 0.798416 2.11413 0.839107 0.347195 30
produce:pandoraShower:LArPandoraShowerCreation 0.29036 0.925729 2.10578 0.864408 0.404739 30
produce:pandoracalo:Calorimetry 0.164327 0.461344 1.03623 0.475686 0.170725 30
produce:pandoracalonosce:Calorimetry 0.147515 0.408621 0.970424 0.434477 0.156666 30
produce:pandorapid:Chi2ParticleID 0.000820806 0.00166425 0.00351967 0.00165554 0.000512579 30
produce:pandoraShowercalo:ShowerCalorimetry 0.245698 0.90707 2.00777 0.843471 0.42131 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.239604 0.890179 2.05521 0.842035 0.420822 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0518474 0.130148 0.21755 0.13406 0.0439086 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0173469 0.043185 0.0688831 0.0462498 0.0139589 30
produce:emtrkmichelid:EmTrackMichelId 12.7951 21.9801 38.0224 22.2938 5.5101 30
[art]:TriggerResults:TriggerResultInserter 2.0368e-05 3.62947e-05 9.1743e-05 3.35635e-05 1.38223e-05 30
end_path:out1:RootOutput 7.314e-06 1.11339e-05 2.8884e-05 1.056e-05 3.6839e-06 30
end_path:michelremoving:michelremoving 0.0133505 0.0694803 0.470648 0.0267624 0.0959083 30
end_path:michelremovinggnocchi:michelremoving 0.011415 0.0679473 0.460305 0.0240413 0.0978184 30
end_path:calibanagnocchi:CalibAnaTree 0.141826 0.359934 0.746076 0.345253 0.147987 30
end_path:out1:RootOutput(write) 0.37397 0.693885 1.08395 0.710622 0.184521 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4942.34 MB
Peak resident set size usage (VmHWM): 2608.86 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2166_dataflow7_datawriter_0_20240908T024601_reco_stage1_reco_stage2_20240926T142959_keepup.root
\tHists: np04hd_raw_run029107_2166_dataflow7_datawriter_0_20240908T024601_reco_stage1_reco_stage2_20240926T142959_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
