Jobsub ID 78788.3@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78788.3@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3453 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 12:38:51 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce14-multicore |
| Last heartbeat | 2024-09-26 14:03:10 |
| From worker node | Hostname | tds211.pic.es |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:39:21 |
| Input files | hd-protodune:np04hd_raw_run029107_3638_dataflow6_datawriter_0_20240909T043055.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:03:10 |
| Saved logs | justin-logs:78788.3-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
TCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 873366 TrkackID 2 startx, y, z 38.1733 344.285 444.794 wireno, peakt tpcno 75 5 5 dist 0min T, max_T5 3631.12
no of hits closeby 0 event 873366 TrkackID 2 startx, y, z 38.1733 344.285 444.794 wireno, peakt tpcno 75 5 5 dist 0min T, max_T5 3631.12
Begin processing the 29th record. run: 29107 subRun: 1 event: 873374 at 26-Sep-2024 15:55:21 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 54716 traces, 54716 summary
Retagger: tagging trace set: gauss with 41629 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41629 traces tagged "gauss"
FrameSaver: q=871344 n=784150 tag=gauss
wclsFrameSaver: saving 54716 traces tagged "wiener"
FrameSaver: q=929169 n=707833 tag=wiener
4171 hits in TPC 0
3060 hits in TPC 1
4145 hits in TPC 2
3633 hits in TPC 3
1627 hits in TPC 4
3314 hits in TPC 5
5338 hits in TPC 6
2612 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1165 XUs and 1039 XVs -> 246 XUVs
C:0 T:2 17575 XUs and 14231 XVs -> 562 XUVs
C:0 T:3 248 XUs and 265 XVs -> 5 XUVs
C:0 T:4 33 XUs and 45 XVs -> 2 XUVs
C:0 T:5 6798 XUs and 4386 XVs -> 153 XUVs
C:0 T:6 11712 XUs and 13277 XVs -> 286 XUVs
C:0 T:7 129 XUs and 169 XVs -> 21 XUVs
1275 XUVs total
690 collection wire objects
1275 potential space points
Neighbour search...
26735 tests to find 16004 neighbours
Iterating with no regularization...
Begin: 3.44211e+06
0 3.22725e+06
1 3.21267e+06
2 3.21055e+06
Now with regularization...
Begin: 3.0156e+06
0 3.01512e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 873374 TrkackID 0 startx, y, z 73.4595 369.6 428.493 wireno, peakt tpcno 154 4151.2 6 dist 403.31min T, max_T2062.37 4171.72
no of hits closeby 1 event 873374 TrkackID 17 startx, y, z 21.0041 587.432 214.828 wireno, peakt tpcno 433 5509.36 2 dist 107.691min T, max_T5293.49 5570.94
no of hits closeby 0 event 873374 TrkackID 0 startx, y, z 73.4595 369.6 428.493 wireno, peakt tpcno 154 4151.2 6 dist 403.31min T, max_T2062.37 4171.72
no of hits closeby 1 event 873374 TrkackID 17 startx, y, z 21.0041 587.432 214.828 wireno, peakt tpcno 433 5509.36 2 dist 107.691min T, max_T5293.49 5570.94
Begin processing the 30th record. run: 29107 subRun: 1 event: 873382 at 26-Sep-2024 15:56:48 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 58903 traces, 58903 summary
Retagger: tagging trace set: gauss with 43033 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 43033 traces tagged "gauss"
FrameSaver: q=901109 n=778904 tag=gauss
wclsFrameSaver: saving 58903 traces tagged "wiener"
FrameSaver: q=1.0051e+06 n=711120 tag=wiener
3782 hits in TPC 0
2636 hits in TPC 1
4934 hits in TPC 2
3572 hits in TPC 3
1401 hits in TPC 4
2652 hits in TPC 5
6607 hits in TPC 6
4236 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 498 XUs and 509 XVs -> 21 XUVs
C:0 T:2 24974 XUs and 56649 XVs -> 2009 XUVs
C:0 T:3 38 XUs and 34 XVs -> 1 XUVs
C:0 T:4 68 XUs and 66 XVs -> 1 XUVs
C:0 T:5 11436 XUs and 54524 XVs -> 1707 XUVs
C:0 T:6 19694 XUs and 50012 XVs -> 852 XUVs
C:0 T:7 112 XUs and 187 XVs -> 1 XUVs
4592 XUVs total
1324 collection wire objects
4592 potential space points
Neighbour search...
302348 tests to find 145764 neighbours
Iterating with no regularization...
Begin: 2.0196e+07
0 1.81856e+07
1 1.80291e+07
2 1.80169e+07
Now with regularization...
Begin: 1.76546e+07
0 1.76504e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 15:58:18 CEST Closed output file "np04hd_raw_run029107_3638_dataflow6_datawriter_0_20240909T043055_reco_stage1_reco_stage2_20240926T135818_keepup.root"
26-Sep-2024 15:58:18 CEST Closed input file "np04hd_raw_run029107_3638_dataflow6_datawriter_0_20240909T043055_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 61.6975 88.7756 123.055 86.5805 11.7745 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000432598 0.00201779 0.00305671 0.00212993 0.000730593 30
produce:fembfilter:PDHDFEMBFilter 0.564658 0.597281 0.644908 0.592962 0.0181297 30
produce:wclsdatahd:WireCellToolkit 40.4085 45.1101 54.0272 44.2193 3.42679 30
produce:gaushit:GausHitFinder 0.350201 0.68339 0.906916 0.668446 0.124314 30
produce:nhitsfilter:NumberOfHitsFilter 0.000180433 0.000351913 0.000648553 0.000350748 8.94283e-05 30
produce:reco3d:SpacePointSolver 0.136725 0.7076 1.74352 0.585226 0.420246 30
produce:hitpdune:DisambigFromSpacePoints 0.121973 0.450521 0.764676 0.444114 0.161285 30
produce:pandora:StandardPandora 3.24118 11.278 21.8337 10.672 3.90652 30
produce:pandoraWriter:StandardPandora 0.0818408 0.133217 0.235678 0.125171 0.0295204 30
produce:pandoraTrack:LArPandoraTrackCreation 0.381299 1.27633 3.32275 1.14591 0.562908 30
produce:pandoraShower:LArPandoraShowerCreation 0.287948 1.43015 5.22928 1.20021 0.866425 30
produce:pandoracalo:Calorimetry 0.224977 0.634679 1.42531 0.575899 0.225574 30
produce:pandoracalonosce:Calorimetry 0.195364 0.581313 1.4982 0.53972 0.245663 30
produce:pandorapid:Chi2ParticleID 0.00120414 0.00170496 0.00295583 0.00164355 0.000377941 30
produce:pandoraShowercalo:ShowerCalorimetry 0.226594 1.30229 4.46206 1.08752 0.741541 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.222757 1.26937 3.80243 1.08431 0.65918 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0713173 0.125348 0.20811 0.120562 0.0256414 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.025258 0.0445494 0.0773351 0.0437572 0.00974471 30
produce:emtrkmichelid:EmTrackMichelId 11.8295 21.7314 34.5805 21.288 4.07613 30
[art]:TriggerResults:TriggerResultInserter 1.9257e-05 3.15045e-05 7.8217e-05 2.6409e-05 1.38189e-05 30
end_path:out1:RootOutput 7.439e-06 1.03102e-05 2.5202e-05 9.5795e-06 3.36282e-06 30
end_path:michelremoving:michelremoving 0.0222952 0.0955664 0.474571 0.0365753 0.104395 30
end_path:michelremovinggnocchi:michelremoving 0.0214781 0.0916496 0.436574 0.0336769 0.0993512 30
end_path:calibanagnocchi:CalibAnaTree 0.179448 0.471534 1.15083 0.422668 0.188627 30
end_path:out1:RootOutput(write) 0.370042 0.674422 1.00656 0.665762 0.125231 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4924.69 MB
Peak resident set size usage (VmHWM): 2580.95 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_3638_dataflow6_datawriter_0_20240909T043055_reco_stage1_reco_stage2_20240926T135818_keepup.root
\tHists: np04hd_raw_run029107_3638_dataflow6_datawriter_0_20240909T043055_reco_stage1_reco_stage2_20240926T135818_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
