Jobsub ID 263339.33@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263339.33@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 10:13:47 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 12:20:18 |
| From worker node | Hostname | cn-leo-025.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:44:44 |
| Input files | hd-protodune:np04hd_raw_run029081_0802_dataflow2_datawriter_0_20240905T053122.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 12:20:18 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
n...
Begin: 1.87232e+07
0 1.87203e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 192730 at 26-Sep-2024 13:52:58 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 75074 traces, 75074 summary
Retagger: tagging trace set: gauss with 54394 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 54394 traces tagged "gauss"
FrameSaver: q=1.19748e+06 n=1002385 tag=gauss
wclsFrameSaver: saving 75074 traces tagged "wiener"
FrameSaver: q=1.29517e+06 n=910539 tag=wiener
5876 hits in TPC 0
4842 hits in TPC 1
4479 hits in TPC 2
3021 hits in TPC 3
2676 hits in TPC 4
6124 hits in TPC 5
7702 hits in TPC 6
4265 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2940 XUs and 4011 XVs -> 749 XUVs
C:0 T:2 14201 XUs and 9679 XVs -> 653 XUVs
C:0 T:3 236 XUs and 262 XVs -> 19 XUVs
C:0 T:4 214 XUs and 387 XVs -> 7 XUVs
C:0 T:5 15482 XUs and 21138 XVs -> 421 XUVs
C:0 T:6 32478 XUs and 44026 XVs -> 6846 XUVs
C:0 T:7 299 XUs and 524 XVs -> 31 XUVs
8726 XUVs total
1339 collection wire objects
8726 potential space points
Neighbour search...
1841470 tests to find 840972 neighbours
Iterating with no regularization...
Begin: 7.36068e+06
0 6.64577e+06
1 6.59898e+06
2 6.58994e+06
3 6.5863e+06
Now with regularization...
Begin: 6.21568e+06
0 6.21094e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 192738 at 26-Sep-2024 13:54:52 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66642 traces, 66642 summary
Retagger: tagging trace set: gauss with 49162 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 49162 traces tagged "gauss"
FrameSaver: q=1.04449e+06 n=898004 tag=gauss
wclsFrameSaver: saving 66642 traces tagged "wiener"
FrameSaver: q=1.1139e+06 n=800630 tag=wiener
1920 hits in TPC 0
2975 hits in TPC 1
8872 hits in TPC 2
4848 hits in TPC 3
1996 hits in TPC 4
4306 hits in TPC 5
5287 hits in TPC 6
3598 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 78 XUs and 114 XVs -> 0 XUVs
C:0 T:2 44007 XUs and 62483 XVs -> 2166 XUVs
C:0 T:3 63 XUs and 69 XVs -> 3 XUVs
C:0 T:4 18 XUs and 4 XVs -> 0 XUVs
C:0 T:5 7139 XUs and 11383 XVs -> 547 XUVs
C:0 T:6 20085 XUs and 39578 XVs -> 1066 XUVs
C:0 T:7 446 XUs and 817 XVs -> 10 XUVs
3792 XUVs total
1575 collection wire objects
3792 potential space points
Neighbour search...
141918 tests to find 83112 neighbours
Iterating with no regularization...
Begin: 7.22706e+06
0 6.55907e+06
1 6.51274e+06
2 6.50554e+06
3 6.50332e+06
Now with regularization...
Begin: 6.03344e+06
0 6.03228e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 192738 TrkackID 15 startx, y, z -107.084 596.193 135.734 wireno, peakt tpcno 109 1283.17 2 dist 0.206909min T, max_T254.576 1634.11
no of hits closeby 0 event 192738 TrkackID 15 startx, y, z -107.084 596.193 135.734 wireno, peakt tpcno 109 1283.17 2 dist 0.206909min T, max_T254.576 1634.11
26-Sep-2024 13:56:50 CEST Closed output file "np04hd_raw_run029081_0802_dataflow2_datawriter_0_20240905T053122_reco_stage1_reco_stage2_20240926T115650_keepup.root"
26-Sep-2024 13:56:50 CEST Closed input file "np04hd_raw_run029081_0802_dataflow2_datawriter_0_20240905T053122_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 79.5023 104.063 145.969 103.384 15.6223 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000585721 0.00250535 0.00804178 0.00160799 0.00209797 30
produce:fembfilter:PDHDFEMBFilter 0.674551 0.738178 0.970861 0.727968 0.0473943 30
produce:wclsdatahd:WireCellToolkit 57.838 61.7419 68.8313 61.7711 2.55337 30
produce:gaushit:GausHitFinder 0.210432 0.558611 0.844691 0.583159 0.142583 30
produce:nhitsfilter:NumberOfHitsFilter 0.00034645 0.000577956 0.00121783 0.000580913 0.000166509 30
produce:reco3d:SpacePointSolver 0.139455 1.17201 3.44587 1.08099 0.777647 30
produce:hitpdune:DisambigFromSpacePoints 0.0838914 0.373112 0.870081 0.382612 0.167468 30
produce:pandora:StandardPandora 3.92305 10.3917 21.4027 9.85056 4.0827 30
produce:pandoraWriter:StandardPandora 0.0988862 0.16334 0.252025 0.168246 0.0395549 30
produce:pandoraTrack:LArPandoraTrackCreation 0.854691 2.54082 4.64658 2.51471 1.11187 30
produce:pandoraShower:LArPandoraShowerCreation 0.76505 3.09902 6.49855 2.97392 1.59302 30
produce:pandoracalo:Calorimetry 0.352372 1.02896 1.82603 1.06771 0.42774 30
produce:pandoracalonosce:Calorimetry 0.320563 0.965263 1.74101 0.967677 0.415343 30
produce:pandorapid:Chi2ParticleID 0.00107544 0.00172037 0.00249708 0.00181011 0.000341439 30
produce:pandoraShowercalo:ShowerCalorimetry 0.529955 2.45077 5.81099 2.43218 1.37295 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.534732 2.38656 5.32164 2.3412 1.30783 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0525987 0.103787 0.16694 0.107892 0.0288755 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0213693 0.0455897 0.0682354 0.0463993 0.0130729 30
produce:emtrkmichelid:EmTrackMichelId 8.18337 14.4018 22.6967 15.0798 3.58273 30
[art]:TriggerResults:TriggerResultInserter 4.1926e-05 6.8539e-05 9.8986e-05 7.0862e-05 1.53344e-05 30
end_path:out1:RootOutput 1.3607e-05 2.09825e-05 3.0764e-05 1.9893e-05 4.31578e-06 30
end_path:michelremoving:michelremoving 0.027772 0.080272 0.241989 0.0602247 0.051967 30
end_path:michelremovinggnocchi:michelremoving 0.0262238 0.0776715 0.235042 0.059456 0.0531255 30
end_path:calibanagnocchi:CalibAnaTree 0.24655 0.718489 1.49044 0.728257 0.316798 30
end_path:out1:RootOutput(write) 0.311505 0.639239 1.06272 0.666916 0.169233 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5783.78 MB
Peak resident set size usage (VmHWM): 2639.22 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0802_dataflow2_datawriter_0_20240905T053122_reco_stage1_reco_stage2_20240926T115650_keepup.root
\tHists: np04hd_raw_run029081_0802_dataflow2_datawriter_0_20240905T053122_reco_stage1_reco_stage2_20240926T115650_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
