Jobsub ID 263763.139@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263763.139@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:49 |
| Site | UK_Sheffield |
| Entry | DUNE_UK_Sheffield_lcgce1 |
| Last heartbeat | 2024-09-26 14:39:44 |
| From worker node | Hostname | wn031.hep |
| cpuinfo | Intel(R) Xeon(R) Gold 6226 CPU @ 2.70GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:26:35 |
| Input files | hd-protodune:np04hd_raw_run029097_0130_dataflow2_datawriter_0_20240906T130412.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:39:44 |
| Saved logs | justin-logs:263763.139-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
0 V bad channels
Finding XUV coincidences...
C:0 T:1 707 XUs and 759 XVs -> 4 XUVs
C:0 T:2 22415 XUs and 7899 XVs -> 366 XUVs
C:0 T:3 97 XUs and 72 XVs -> 3 XUVs
C:0 T:4 28 XUs and 39 XVs -> 1 XUVs
C:0 T:5 14887 XUs and 17864 XVs -> 411 XUVs
C:0 T:6 32010 XUs and 30318 XVs -> 1184 XUVs
C:0 T:7 101 XUs and 165 XVs -> 1 XUVs
1970 XUVs total
1242 collection wire objects
1970 potential space points
Neighbour search...
34970 tests to find 20140 neighbours
Iterating with no regularization...
Begin: 1.0819e+07
0 1.01844e+07
1 1.01573e+07
2 1.01549e+07
Now with regularization...
Begin: 9.6969e+06
0 9.69613e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 31426 at 26-Sep-2024 15:30:46 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 77873 traces, 77873 summary
Retagger: tagging trace set: gauss with 57632 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 57632 traces tagged "gauss"
FrameSaver: q=1.29277e+06 n=1052007 tag=gauss
wclsFrameSaver: saving 77873 traces tagged "wiener"
FrameSaver: q=1.3964e+06 n=954348 tag=wiener
4432 hits in TPC 0
4225 hits in TPC 1
7513 hits in TPC 2
5107 hits in TPC 3
4122 hits in TPC 4
6730 hits in TPC 5
5047 hits in TPC 6
3378 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 5598 XUs and 3236 XVs -> 66 XUVs
C:0 T:2 23765 XUs and 23935 XVs -> 763 XUVs
C:0 T:3 477 XUs and 666 XVs -> 12 XUVs
C:0 T:4 53 XUs and 79 XVs -> 1 XUVs
C:0 T:5 29478 XUs and 53604 XVs -> 5078 XUVs
C:0 T:6 19308 XUs and 12900 XVs -> 527 XUVs
C:0 T:7 1129 XUs and 752 XVs -> 25 XUVs
6472 XUVs total
1663 collection wire objects
6472 potential space points
Neighbour search...
634680 tests to find 333588 neighbours
Iterating with no regularization...
Begin: 1.03694e+07
0 9.26899e+06
1 9.19032e+06
2 9.17771e+06
3 9.17408e+06
Now with regularization...
Begin: 8.57667e+06
0 8.57358e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 4 event 31426 TrkackID 0 startx, y, z 196.581 459.295 292.479 wireno, peakt tpcno 191 1130.67 5 dist 2.20992min T, max_T6.10375 2502.96
no of hits closeby 4 event 31426 TrkackID 0 startx, y, z 196.581 459.295 292.479 wireno, peakt tpcno 191 1130.67 5 dist 2.20992min T, max_T6.10375 2502.96
Begin processing the 30th record. run: 29097 subRun: 1 event: 31434 at 26-Sep-2024 15:32:25 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 69179 traces, 69179 summary
Retagger: tagging trace set: gauss with 50036 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50036 traces tagged "gauss"
FrameSaver: q=1.10759e+06 n=936397 tag=gauss
wclsFrameSaver: saving 69179 traces tagged "wiener"
FrameSaver: q=1.19303e+06 n=848509 tag=wiener
3602 hits in TPC 0
2736 hits in TPC 1
4780 hits in TPC 2
2511 hits in TPC 3
3461 hits in TPC 4
4367 hits in TPC 5
8424 hits in TPC 6
7020 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 196 XUs and 420 XVs -> 0 XUVs
C:0 T:2 28066 XUs and 45075 XVs -> 1497 XUVs
C:0 T:3 52 XUs and 61 XVs -> 1 XUVs
C:0 T:4 339 XUs and 580 XVs -> 29 XUVs
C:0 T:5 9142 XUs and 18531 XVs -> 638 XUVs
C:0 T:6 56339 XUs and 67860 XVs -> 4493 XUVs
C:0 T:7 666 XUs and 870 XVs -> 37 XUVs
6695 XUVs total
1781 collection wire objects
6695 potential space points
Neighbour search...
451939 tests to find 246202 neighbours
Iterating with no regularization...
Begin: 8.95891e+06
0 8.24951e+06
1 8.18829e+06
2 8.17734e+06
3 8.17318e+06
Now with regularization...
Begin: 7.45966e+06
0 7.45666e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:34:01 BST Closed output file "np04hd_raw_run029097_0130_dataflow2_datawriter_0_20240906T130412_reco_stage1_reco_stage2_20240926T143401_keepup.root"
26-Sep-2024 15:34:01 BST Closed input file "np04hd_raw_run029097_0130_dataflow2_datawriter_0_20240906T130412_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 65.2748 80.9952 104.472 80.824 10.147 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000347856 0.00149244 0.00477533 0.000914903 0.00121118 30
produce:fembfilter:PDHDFEMBFilter 0.590708 0.603476 0.632259 0.601941 0.00935582 30
produce:wclsdatahd:WireCellToolkit 43.5823 46.183 47.9812 46.4574 1.20719 30
produce:gaushit:GausHitFinder 0.392002 0.616333 0.88024 0.635196 0.122134 30
produce:nhitsfilter:NumberOfHitsFilter 0.000278129 0.000382675 0.000650916 0.000375922 7.37562e-05 30
produce:reco3d:SpacePointSolver 0.175375 0.952454 3.31009 0.730571 0.684412 30
produce:hitpdune:DisambigFromSpacePoints 0.141336 0.466328 0.818443 0.483907 0.174933 30
produce:pandora:StandardPandora 4.60803 9.61222 16.0904 9.12509 3.13432 30
produce:pandoraWriter:StandardPandora 0.0922563 0.140411 0.210626 0.137092 0.0292016 30
produce:pandoraTrack:LArPandoraTrackCreation 0.868967 1.95817 4.68722 1.81831 0.869314 30
produce:pandoraShower:LArPandoraShowerCreation 0.886176 2.20185 5.42305 1.90365 1.11222 30
produce:pandoracalo:Calorimetry 0.381141 0.803433 1.66798 0.773977 0.289382 30
produce:pandoracalonosce:Calorimetry 0.323143 0.748894 1.55614 0.695645 0.282166 30
produce:pandorapid:Chi2ParticleID 0.00111314 0.00163379 0.00301324 0.00155891 0.000363777 30
produce:pandoraShowercalo:ShowerCalorimetry 0.624055 1.84319 3.88252 1.77292 0.762432 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.604714 1.82116 3.85785 1.7809 0.766525 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0780142 0.112296 0.156686 0.111067 0.0212024 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0326677 0.0457499 0.0598497 0.045149 0.00857846 30
produce:emtrkmichelid:EmTrackMichelId 7.6047 11.3565 15.3024 11.559 2.05482 30
[art]:TriggerResults:TriggerResultInserter 2.8059e-05 4.36224e-05 8.7746e-05 4.07095e-05 1.44186e-05 30
end_path:out1:RootOutput 4.727e-06 1.12056e-05 4.2737e-05 9.9655e-06 6.30339e-06 30
end_path:michelremoving:michelremoving 0.0278255 0.0715466 0.280194 0.0481995 0.051786 30
end_path:michelremovinggnocchi:michelremoving 0.0252381 0.0694133 0.277554 0.043783 0.0533245 30
end_path:calibanagnocchi:CalibAnaTree 0.28018 0.553327 0.94421 0.543141 0.186479 30
end_path:out1:RootOutput(write) 0.452356 0.638002 0.833269 0.624533 0.120391 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5874.11 MB
Peak resident set size usage (VmHWM): 2706.26 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0130_dataflow2_datawriter_0_20240906T130412_reco_stage1_reco_stage2_20240926T143401_keepup.root
\tHists: np04hd_raw_run029097_0130_dataflow2_datawriter_0_20240906T130412_reco_stage1_reco_stage2_20240926T143401_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
