Jobsub ID 78692.100@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78692.100@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 11:28:50 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 14:19:01 |
| From worker node | Hostname | cn-608-05-08.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:18:01 |
| Input files | hd-protodune:np04hd_raw_run029081_0685_dataflow3_datawriter_0_20240905T035521.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:19:01 |
| Saved logs | justin-logs:78692.100-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
nd 18059 XVs -> 770 XUVs
C:0 T:3 902 XUs and 1098 XVs -> 15 XUVs
C:0 T:4 209 XUs and 240 XVs -> 17 XUVs
C:0 T:5 45802 XUs and 61077 XVs -> 1785 XUVs
C:0 T:6 12474 XUs and 10733 XVs -> 296 XUVs
C:0 T:7 0 XUs and 0 XVs -> 0 XUVs
2898 XUVs total
1413 collection wire objects
2898 potential space points
Neighbour search...
204958 tests to find 92526 neighbours
Iterating with no regularization...
Begin: 1.16376e+07
0 1.10534e+07
1 1.10355e+07
2 1.10333e+07
Now with regularization...
Begin: 1.04553e+07
0 1.04543e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 164627 TrkackID 10 startx, y, z -9.80445 588.456 320.597 wireno, peakt tpcno 459 2242.03 6 dist 0.401283min T, max_T2241.39 3072.89
no of hits closeby 0 event 164627 TrkackID 10 startx, y, z -9.80445 588.456 320.597 wireno, peakt tpcno 459 2242.03 6 dist 0.401283min T, max_T2241.39 3072.89
Begin processing the 29th record. run: 29081 subRun: 1 event: 164635 at 26-Sep-2024 16:11:36 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 56404 traces, 56404 summary
Retagger: tagging trace set: gauss with 41823 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41823 traces tagged "gauss"
FrameSaver: q=923447 n=767624 tag=gauss
wclsFrameSaver: saving 56404 traces tagged "wiener"
FrameSaver: q=977924 n=686803 tag=wiener
1878 hits in TPC 0
1656 hits in TPC 1
4329 hits in TPC 2
918 hits in TPC 3
3704 hits in TPC 4
5317 hits in TPC 5
6793 hits in TPC 6
4176 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 45 XUs and 64 XVs -> 1 XUVs
C:0 T:2 18284 XUs and 29390 XVs -> 1096 XUVs
C:0 T:3 259 XUs and 453 XVs -> 22 XUVs
C:0 T:4 38 XUs and 89 XVs -> 2 XUVs
C:0 T:5 16459 XUs and 15858 XVs -> 654 XUVs
C:0 T:6 37172 XUs and 33801 XVs -> 1658 XUVs
C:0 T:7 165 XUs and 281 XVs -> 3 XUVs
3436 XUVs total
1575 collection wire objects
3436 potential space points
Neighbour search...
142968 tests to find 75018 neighbours
Iterating with no regularization...
Begin: 8.99725e+06
0 8.50853e+06
1 8.48701e+06
2 8.48347e+06
Now with regularization...
Begin: 7.92629e+06
0 7.92487e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 164643 at 26-Sep-2024 16:14:12 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 95952 traces, 95952 summary
Retagger: tagging trace set: gauss with 69493 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 69493 traces tagged "gauss"
FrameSaver: q=1.52018e+06 n=1269198 tag=gauss
wclsFrameSaver: saving 95952 traces tagged "wiener"
FrameSaver: q=1.66938e+06 n=1146405 tag=wiener
4906 hits in TPC 0
6265 hits in TPC 1
7237 hits in TPC 2
5050 hits in TPC 3
4840 hits in TPC 4
8357 hits in TPC 5
9619 hits in TPC 6
4669 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 8883 XUs and 8796 XVs -> 165 XUVs
C:0 T:2 61667 XUs and 47404 XVs -> 652 XUVs
C:0 T:3 86 XUs and 74 XVs -> 1 XUVs
C:0 T:4 148 XUs and 233 XVs -> 7 XUVs
C:0 T:5 35683 XUs and 33662 XVs -> 1344 XUVs
C:0 T:6 62013 XUs and 60112 XVs -> 2370 XUVs
C:0 T:7 886 XUs and 821 XVs -> 21 XUVs
4560 XUVs total
1892 collection wire objects
4560 potential space points
Neighbour search...
312420 tests to find 147494 neighbours
Iterating with no regularization...
Begin: 1.41621e+07
0 1.24376e+07
1 1.23282e+07
2 1.2318e+07
Now with regularization...
Begin: 1.16954e+07
0 1.1691e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 16:18:29 CEST Closed output file "np04hd_raw_run029081_0685_dataflow3_datawriter_0_20240905T035521_reco_stage1_reco_stage2_20240926T141829_keepup.root"
26-Sep-2024 16:18:29 CEST Closed input file "np04hd_raw_run029081_0685_dataflow3_datawriter_0_20240905T035521_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 117.893 180.233 339.29 173.606 47.1789 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000665059 0.00279883 0.0046682 0.00258604 0.00128641 30
produce:fembfilter:PDHDFEMBFilter 0.687982 1.04265 1.12499 1.06791 0.10237 30
produce:wclsdatahd:WireCellToolkit 64.0383 76.3936 84.9392 76.8208 4.30673 30
produce:gaushit:GausHitFinder 0.431786 1.18795 2.10595 1.1513 0.410012 30
produce:nhitsfilter:NumberOfHitsFilter 0.00034228 0.000550261 0.00107237 0.000511548 0.000158962 30
produce:reco3d:SpacePointSolver 0.211748 1.60773 6.09195 1.30973 1.43849 30
produce:hitpdune:DisambigFromSpacePoints 0.20674 1.24512 2.91137 1.11341 0.770331 30
produce:pandora:StandardPandora 5.10682 25.2475 83.5123 19.2938 16.5996 30
produce:pandoraWriter:StandardPandora 0.121103 0.258263 0.443828 0.254432 0.0754942 30
produce:pandoraTrack:LArPandoraTrackCreation 0.796915 3.27021 9.65156 2.81543 1.80804 30
produce:pandoraShower:LArPandoraShowerCreation 0.710085 4.02652 16.6433 2.93596 3.2051 30
produce:pandoracalo:Calorimetry 0.353979 1.49704 4.53765 1.35378 0.80421 30
produce:pandoracalonosce:Calorimetry 0.304381 1.40637 4.64765 1.29719 0.820194 30
produce:pandorapid:Chi2ParticleID 0.00146537 0.00331029 0.00619885 0.00325623 0.000942445 30
produce:pandoraShowercalo:ShowerCalorimetry 0.454023 3.49669 13.6041 2.76294 2.53455 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.466532 3.37719 13.173 2.7653 2.43151 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0788849 0.247487 0.453194 0.238215 0.0746797 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0298577 0.0939199 0.166795 0.0958109 0.0279721 30
produce:emtrkmichelid:EmTrackMichelId 31.5152 52.9261 93.2451 50.2476 15.0601 30
[art]:TriggerResults:TriggerResultInserter 4.5027e-05 6.88337e-05 0.000117496 6.88125e-05 1.62438e-05 30
end_path:out1:RootOutput 7.832e-06 1.13206e-05 3.3162e-05 9.914e-06 4.74302e-06 30
end_path:michelremoving:michelremoving 0.0366033 0.155835 0.518738 0.0807285 0.138843 30
end_path:michelremovinggnocchi:michelremoving 0.034049 0.147538 0.510075 0.0722558 0.134098 30
end_path:calibanagnocchi:CalibAnaTree 0.351918 1.10316 3.32093 0.952564 0.628535 30
end_path:out1:RootOutput(write) 0.663265 1.32106 2.08376 1.31446 0.322214 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4937.48 MB
Peak resident set size usage (VmHWM): 2363.96 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0685_dataflow3_datawriter_0_20240905T035521_reco_stage1_reco_stage2_20240926T141829_keepup.root
\tHists: np04hd_raw_run029081_0685_dataflow3_datawriter_0_20240905T035521_reco_stage1_reco_stage2_20240926T141829_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
