Jobsub ID 263931.97@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263931.97@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 14:23:56 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 16:33:29 |
| From worker node | Hostname | cn-314-02-01.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 14:25:22 |
| Input files | hd-protodune:np04hd_raw_run029107_0008_dataflow4_datawriter_0_20240906T163807.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 16:33:29 |
| Saved logs | justin-logs:263931.97-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
59 XUVs
C:0 T:7 186 XUs and 141 XVs -> 6 XUVs
2444 XUVs total
1022 collection wire objects
2444 potential space points
Neighbour search...
117460 tests to find 56740 neighbours
Iterating with no regularization...
Begin: 9.59788e+06
0 9.30867e+06
1 9.28706e+06
2 9.28418e+06
Now with regularization...
Begin: 8.77848e+06
0 8.77754e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 2172 at 26-Sep-2024 18:26:56 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 63795 traces, 63795 summary
Retagger: tagging trace set: gauss with 49351 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49351 traces tagged "gauss"
FrameSaver: q=1.06823e+06 n=953560 tag=gauss
wclsFrameSaver: saving 63795 traces tagged "wiener"
FrameSaver: q=1.1349e+06 n=868394 tag=wiener
3412 hits in TPC 0
3825 hits in TPC 1
6110 hits in TPC 2
4658 hits in TPC 3
3351 hits in TPC 4
7401 hits in TPC 5
4315 hits in TPC 6
2983 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 438 XUs and 151 XVs -> 1 XUVs
C:0 T:2 19264 XUs and 26631 XVs -> 916 XUVs
C:0 T:3 279 XUs and 560 XVs -> 18 XUVs
C:0 T:4 623 XUs and 801 XVs -> 28 XUVs
C:0 T:5 22151 XUs and 19242 XVs -> 633 XUVs
C:0 T:6 11411 XUs and 10382 XVs -> 377 XUVs
C:0 T:7 2714 XUs and 3249 XVs -> 95 XUVs
2068 XUVs total
1067 collection wire objects
2068 potential space points
Neighbour search...
49144 tests to find 30092 neighbours
Iterating with no regularization...
Begin: 3.78443e+06
0 3.42944e+06
1 3.38943e+06
2 3.38463e+06
3 3.38307e+06
Now with regularization...
Begin: 3.11098e+06
0 3.11019e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 5 event 2172 TrkackID 0 startx, y, z 82.6446 599.891 81.567 wireno, peakt tpcno 467 2053.33 5 dist 0min T, max_T2047.35 3394.64
no of hits closeby 5 event 2172 TrkackID 0 startx, y, z 82.6446 599.891 81.567 wireno, peakt tpcno 467 2053.33 5 dist 0min T, max_T2047.35 3394.64
Begin processing the 30th record. run: 29107 subRun: 1 event: 2180 at 26-Sep-2024 18:30:11 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 57069 traces, 57069 summary
Retagger: tagging trace set: gauss with 41263 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 41263 traces tagged "gauss"
FrameSaver: q=958664 n=769047 tag=gauss
wclsFrameSaver: saving 57069 traces tagged "wiener"
FrameSaver: q=1.03076e+06 n=691183 tag=wiener
1975 hits in TPC 0
3075 hits in TPC 1
3287 hits in TPC 2
2300 hits in TPC 3
4841 hits in TPC 4
5692 hits in TPC 5
4884 hits in TPC 6
3674 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 785 XUs and 1078 XVs -> 42 XUVs
C:0 T:2 8237 XUs and 7805 XVs -> 1048 XUVs
C:0 T:3 335 XUs and 672 XVs -> 3 XUVs
C:0 T:4 95 XUs and 81 XVs -> 12 XUVs
C:0 T:5 22056 XUs and 18342 XVs -> 394 XUVs
C:0 T:6 13639 XUs and 9148 XVs -> 707 XUVs
C:0 T:7 963 XUs and 696 XVs -> 17 XUVs
2223 XUVs total
1142 collection wire objects
2223 potential space points
Neighbour search...
45851 tests to find 27712 neighbours
Iterating with no regularization...
Begin: 7.84678e+06
0 7.51794e+06
1 7.48903e+06
2 7.48468e+06
Now with regularization...
Begin: 6.97003e+06
0 6.96893e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 18:32:59 CEST Closed output file "np04hd_raw_run029107_0008_dataflow4_datawriter_0_20240906T163807_reco_stage1_reco_stage2_20240926T163259_keepup.root"
26-Sep-2024 18:33:00 CEST Closed input file "np04hd_raw_run029107_0008_dataflow4_datawriter_0_20240906T163807_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 121.323 192.859 296.384 187.335 35.9549 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000681113 0.00343842 0.00671625 0.00323687 0.00145922 30
produce:fembfilter:PDHDFEMBFilter 1.02269 1.06711 1.14499 1.06167 0.0253566 30
produce:wclsdatahd:WireCellToolkit 74.5062 79.8436 83.4578 79.9696 2.36885 30
produce:gaushit:GausHitFinder 0.616913 1.28332 1.98142 1.31503 0.270044 30
produce:nhitsfilter:NumberOfHitsFilter 0.000358601 0.000634174 0.00106235 0.000616703 0.000166012 30
produce:reco3d:SpacePointSolver 0.255361 1.72519 4.15192 1.38964 1.14372 30
produce:hitpdune:DisambigFromSpacePoints 0.330911 1.30064 3.19127 1.01137 0.761769 30
produce:pandora:StandardPandora 6.71092 27.126 65.225 25.3263 12.698 30
produce:pandoraWriter:StandardPandora 0.148123 0.276963 0.441251 0.263797 0.0638065 30
produce:pandoraTrack:LArPandoraTrackCreation 1.30376 4.574 8.89531 3.92335 1.84141 30
produce:pandoraShower:LArPandoraShowerCreation 1.53786 5.39199 12.2817 4.8552 2.39345 30
produce:pandoracalo:Calorimetry 0.687742 2.01506 3.74837 1.8058 0.754649 30
produce:pandoracalonosce:Calorimetry 0.620798 1.9134 3.82079 1.74643 0.757249 30
produce:pandorapid:Chi2ParticleID 0.00223044 0.00372512 0.00575831 0.00363576 0.000830645 30
produce:pandoraShowercalo:ShowerCalorimetry 1.27924 4.5733 9.72108 4.35644 2.01978 30
produce:pandoraShowercalonosce:ShowerCalorimetry 1.23793 4.40205 9.41679 4.20523 1.94699 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.151584 0.24508 0.350533 0.243053 0.053006 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0636244 0.101315 0.140326 0.100388 0.020484 30
produce:emtrkmichelid:EmTrackMichelId 29.2763 53.5453 84.5135 52.8682 11.8258 30
[art]:TriggerResults:TriggerResultInserter 6.3728e-05 8.07751e-05 0.000147501 7.53935e-05 1.87599e-05 30
end_path:out1:RootOutput 1.2674e-05 1.83324e-05 3.369e-05 1.78785e-05 4.36011e-06 30
end_path:michelremoving:michelremoving 0.0445176 0.178752 0.667884 0.0972036 0.143927 30
end_path:michelremovinggnocchi:michelremoving 0.0453863 0.172601 0.646239 0.0918947 0.141114 30
end_path:calibanagnocchi:CalibAnaTree 0.470426 1.54585 2.97662 1.36997 0.619321 30
end_path:out1:RootOutput(write) 0.741136 1.33193 1.98922 1.34219 0.286263 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4913.13 MB
Peak resident set size usage (VmHWM): 2338.84 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0008_dataflow4_datawriter_0_20240906T163807_reco_stage1_reco_stage2_20240926T163259_keepup.root
\tHists: np04hd_raw_run029107_0008_dataflow4_datawriter_0_20240906T163807_reco_stage1_reco_stage2_20240926T163259_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
