Jobsub ID 263865.0@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263865.0@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:56:43 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:53:27 |
| From worker node | Hostname | cn-314-04-07.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:57:25 |
| Input files | hd-protodune:np04hd_raw_run029097_0148_dataflow0_datawriter_0_20240906T133438.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:53:27 |
| Saved logs | justin-logs:263865.0-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
in TPC 4
5295 hits in TPC 5
2372 hits in TPC 6
2550 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 163 XUs and 206 XVs -> 6 XUVs
C:0 T:2 8883 XUs and 7679 XVs -> 249 XUVs
C:0 T:3 0 XUs and 0 XVs -> 0 XUVs
C:0 T:4 76 XUs and 93 XVs -> 0 XUVs
C:0 T:5 14844 XUs and 15771 XVs -> 429 XUVs
C:0 T:6 8715 XUs and 7709 XVs -> 403 XUVs
C:0 T:7 9 XUs and 29 XVs -> 1 XUVs
1088 XUVs total
706 collection wire objects
1088 potential space points
Neighbour search...
23396 tests to find 11222 neighbours
Iterating with no regularization...
Begin: 2.42894e+06
0 2.29696e+06
1 2.28794e+06
2 2.28724e+06
Now with regularization...
Begin: 2.10129e+06
0 2.10107e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 35800 at 26-Sep-2024 17:46:34 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 63254 traces, 63254 summary
Retagger: tagging trace set: gauss with 46927 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 46927 traces tagged "gauss"
FrameSaver: q=991382 n=866560 tag=gauss
wclsFrameSaver: saving 63254 traces tagged "wiener"
FrameSaver: q=1.06365e+06 n=784088 tag=wiener
3299 hits in TPC 0
1598 hits in TPC 1
6632 hits in TPC 2
4068 hits in TPC 3
1347 hits in TPC 4
2634 hits in TPC 5
9613 hits in TPC 6
4773 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 12 XUs and 9 XVs -> 0 XUVs
C:0 T:2 36943 XUs and 34177 XVs -> 1130 XUVs
C:0 T:3 201 XUs and 243 XVs -> 0 XUVs
C:0 T:4 34 XUs and 38 XVs -> 4 XUVs
C:0 T:5 3778 XUs and 3201 XVs -> 118 XUVs
C:0 T:6 39774 XUs and 48194 XVs -> 2504 XUVs
C:0 T:7 52 XUs and 77 XVs -> 3 XUVs
3759 XUVs total
1478 collection wire objects
3759 potential space points
Neighbour search...
144473 tests to find 85418 neighbours
Iterating with no regularization...
Begin: 4.95262e+06
0 4.44292e+06
1 4.38392e+06
2 4.37312e+06
3 4.37058e+06
Now with regularization...
Begin: 3.94041e+06
0 3.9391e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 35808 at 26-Sep-2024 17:50:00 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 72151 traces, 72151 summary
Retagger: tagging trace set: gauss with 52281 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 52281 traces tagged "gauss"
FrameSaver: q=1.14598e+06 n=914987 tag=gauss
wclsFrameSaver: saving 72151 traces tagged "wiener"
FrameSaver: q=1.23529e+06 n=819144 tag=wiener
3926 hits in TPC 0
4050 hits in TPC 1
6372 hits in TPC 2
4771 hits in TPC 3
3973 hits in TPC 4
4754 hits in TPC 5
4168 hits in TPC 6
3061 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 4350 XUs and 2473 XVs -> 97 XUVs
C:0 T:2 38472 XUs and 41353 XVs -> 1852 XUVs
C:0 T:3 208 XUs and 251 XVs -> 1 XUVs
C:0 T:4 43 XUs and 61 XVs -> 2 XUVs
C:0 T:5 17861 XUs and 10470 XVs -> 388 XUVs
C:0 T:6 14594 XUs and 21567 XVs -> 1103 XUVs
3443 XUVs total
1682 collection wire objects
3443 potential space points
Neighbour search...
120187 tests to find 70556 neighbours
Iterating with no regularization...
Begin: 7.08103e+06
0 6.62325e+06
1 6.59052e+06
2 6.58705e+06
Now with regularization...
Begin: 6.01498e+06
0 6.01391e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 17:52:57 CEST Closed output file "np04hd_raw_run029097_0148_dataflow0_datawriter_0_20240906T133438_reco_stage1_reco_stage2_20240926T155257_keepup.root"
26-Sep-2024 17:52:57 CEST Closed input file "np04hd_raw_run029097_0148_dataflow0_datawriter_0_20240906T133438_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 121.793 173.23 252.921 169.454 30.8885 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000729483 0.00320645 0.00605501 0.00356521 0.00146341 30
produce:fembfilter:PDHDFEMBFilter 1.03737 1.07597 1.11501 1.07052 0.0227618 30
produce:wclsdatahd:WireCellToolkit 72.7287 77.2185 82.7712 77.7618 2.35988 30
produce:gaushit:GausHitFinder 0.61021 1.17206 1.8475 1.17041 0.281974 30
produce:nhitsfilter:NumberOfHitsFilter 0.000359021 0.000585549 0.00141713 0.000554391 0.000201046 30
produce:reco3d:SpacePointSolver 0.396948 1.27209 3.61529 1.19719 0.713073 30
produce:hitpdune:DisambigFromSpacePoints 0.366257 1.14949 3.40551 1.12084 0.698833 30
produce:pandora:StandardPandora 6.8477 22.3865 46.1118 19.5094 9.66586 30
produce:pandoraWriter:StandardPandora 0.142828 0.252275 0.39309 0.246155 0.0662267 30
produce:pandoraTrack:LArPandoraTrackCreation 1.13022 2.66972 6.61995 2.62998 1.20636 30
produce:pandoraShower:LArPandoraShowerCreation 0.758527 3.11698 9.80608 2.7869 1.8519 30
produce:pandoracalo:Calorimetry 0.609346 1.33216 3.0702 1.26036 0.535877 30
produce:pandoracalonosce:Calorimetry 0.559582 1.23548 2.87325 1.11409 0.531818 30
produce:pandorapid:Chi2ParticleID 0.0022014 0.00322622 0.00511695 0.00294289 0.000732058 30
produce:pandoraShowercalo:ShowerCalorimetry 0.696553 2.94883 9.36328 2.61065 1.65418 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.678839 2.8685 8.57175 2.5279 1.56206 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.155164 0.261817 0.465135 0.245517 0.078615 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0610626 0.0980761 0.158832 0.0956873 0.0245147 30
produce:emtrkmichelid:EmTrackMichelId 29.2498 51.5838 79.9349 52.0374 12.0083 30
[art]:TriggerResults:TriggerResultInserter 3.1547e-05 6.40364e-05 0.000129837 6.4239e-05 1.85213e-05 30
end_path:out1:RootOutput 9.512e-06 1.31721e-05 3.4737e-05 1.1868e-05 4.76509e-06 30
end_path:michelremoving:michelremoving 0.0313747 0.119631 0.344626 0.0701253 0.0962365 30
end_path:michelremovinggnocchi:michelremoving 0.0276529 0.111765 0.340878 0.0636056 0.0969676 30
end_path:calibanagnocchi:CalibAnaTree 0.302143 0.915071 2.40896 0.818335 0.414404 30
end_path:out1:RootOutput(write) 0.797425 1.2934 1.98584 1.32096 0.285479 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4940.02 MB
Peak resident set size usage (VmHWM): 2288.84 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0148_dataflow0_datawriter_0_20240906T133438_reco_stage1_reco_stage2_20240926T155257_keepup.root
\tHists: np04hd_raw_run029097_0148_dataflow0_datawriter_0_20240906T133438_reco_stage1_reco_stage2_20240926T155257_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
