Jobsub ID 263851.2@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263851.2@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3449 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:52:03 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 15:44:06 |
| From worker node | Hostname | wn-204-13-05-03-a.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2618L v4 @ 2.20GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:52:53 |
| Input files | hd-protodune:np04hd_raw_run029107_0794_dataflow5_datawriter_0_20240907T035156.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 15:44:06 |
| Saved logs | justin-logs:263851.2-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
mary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 53366 traces tagged "gauss"
FrameSaver: q=1.22471e+06 n=971988 tag=gauss
wclsFrameSaver: saving 74746 traces tagged "wiener"
FrameSaver: q=1.35022e+06 n=883452 tag=wiener
5789 hits in TPC 0
5584 hits in TPC 1
4210 hits in TPC 2
2260 hits in TPC 3
3834 hits in TPC 4
9305 hits in TPC 5
5455 hits in TPC 6
2508 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1579 XUs and 2940 XVs -> 119 XUVs
C:0 T:2 10908 XUs and 8493 XVs -> 399 XUVs
C:0 T:3 210 XUs and 157 XVs -> 4 XUVs
C:0 T:4 195 XUs and 594 XVs -> 11 XUVs
C:0 T:5 60578 XUs and 91510 XVs -> 2126 XUVs
C:0 T:6 10635 XUs and 12803 XVs -> 1350 XUVs
C:0 T:7 304 XUs and 351 XVs -> 7 XUVs
4016 XUVs total
1360 collection wire objects
4016 potential space points
Neighbour search...
136876 tests to find 81626 neighbours
Iterating with no regularization...
Begin: 3.17553e+07
0 2.82266e+07
1 2.8041e+07
2 2.80306e+07
Now with regularization...
Begin: 2.70168e+07
0 2.70122e+07
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29107 subRun: 1 event: 190813 at 26-Sep-2024 17:39:41 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 41819 traces, 41819 summary
Retagger: tagging trace set: gauss with 32949 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 32949 traces tagged "gauss"
FrameSaver: q=655665 n=649807 tag=gauss
wclsFrameSaver: saving 41819 traces tagged "wiener"
FrameSaver: q=692005 n=592444 tag=wiener
2036 hits in TPC 0
3792 hits in TPC 1
4689 hits in TPC 2
4208 hits in TPC 3
1562 hits in TPC 4
2639 hits in TPC 5
2694 hits in TPC 6
1324 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 91 XUs and 34 XVs -> 2 XUVs
C:0 T:2 14267 XUs and 13092 XVs -> 428 XUVs
C:0 T:3 37 XUs and 56 XVs -> 1 XUVs
C:0 T:4 61 XUs and 100 XVs -> 1 XUVs
C:0 T:5 4700 XUs and 2104 XVs -> 47 XUVs
C:0 T:6 5164 XUs and 4252 XVs -> 176 XUVs
C:0 T:7 34 XUs and 53 XVs -> 1 XUVs
656 XUVs total
508 collection wire objects
656 potential space points
Neighbour search...
10568 tests to find 5298 neighbours
Iterating with no regularization...
Begin: 4.10764e+06
0 4.02044e+06
1 4.01796e+06
Now with regularization...
Begin: 3.77596e+06
0 3.7757e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 190821 at 26-Sep-2024 17:41:34 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 35221 traces, 35221 summary
Retagger: tagging trace set: gauss with 26600 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 26600 traces tagged "gauss"
FrameSaver: q=584439 n=493505 tag=gauss
wclsFrameSaver: saving 35221 traces tagged "wiener"
FrameSaver: q=633296 n=452030 tag=wiener
2782 hits in TPC 0
1617 hits in TPC 1
2864 hits in TPC 2
1896 hits in TPC 3
1143 hits in TPC 4
1656 hits in TPC 5
3363 hits in TPC 6
2849 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3 XUs and 9 XVs -> 0 XUVs
C:0 T:2 6182 XUs and 5197 XVs -> 252 XUVs
C:0 T:3 55 XUs and 74 XVs -> 1 XUVs
C:0 T:4 19 XUs and 47 XVs -> 3 XUVs
C:0 T:5 1951 XUs and 1750 XVs -> 183 XUVs
C:0 T:6 25709 XUs and 16936 XVs -> 690 XUVs
C:0 T:7 450 XUs and 441 XVs -> 7 XUVs
1136 XUVs total
570 collection wire objects
1136 potential space points
Neighbour search...
23722 tests to find 14958 neighbours
Iterating with no regularization...
Begin: 1.54797e+07
0 1.41671e+07
1 1.41513e+07
2 1.41494e+07
Now with regularization...
Begin: 1.38398e+07
0 1.3839e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 17:43:39 CEST Closed output file "np04hd_raw_run029107_0794_dataflow5_datawriter_0_20240907T035156_reco_stage1_reco_stage2_20240926T154339_keepup.root"
26-Sep-2024 17:43:39 CEST Closed input file "np04hd_raw_run029107_0794_dataflow5_datawriter_0_20240907T035156_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 113.449 153.355 197.144 154.882 21.7283 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000823777 0.00383223 0.0275873 0.00331858 0.00456961 30
produce:fembfilter:PDHDFEMBFilter 0.854712 0.997701 1.43352 0.891073 0.18086 30
produce:wclsdatahd:WireCellToolkit 67.7519 73.0039 81.1087 71.7614 3.86775 30
produce:gaushit:GausHitFinder 0.495474 1.02903 1.7466 1.01194 0.321862 30
produce:nhitsfilter:NumberOfHitsFilter 0.000263346 0.000455624 0.000730592 0.000441974 0.000105414 30
produce:reco3d:SpacePointSolver 0.197887 1.47635 4.74782 1.04001 1.24334 30
produce:hitpdune:DisambigFromSpacePoints 0.177805 0.818287 2.14057 0.73516 0.465704 30
produce:pandora:StandardPandora 9.72322 18.9401 37.4247 18.2415 7.10504 30
produce:pandoraWriter:StandardPandora 0.128054 0.212513 0.344112 0.218667 0.0589371 30
produce:pandoraTrack:LArPandoraTrackCreation 0.794151 2.18894 4.07514 2.10764 0.860999 30
produce:pandoraShower:LArPandoraShowerCreation 0.806411 2.65592 5.34752 2.34169 1.25325 30
produce:pandoracalo:Calorimetry 0.456181 1.01289 1.56668 1.05957 0.317457 30
produce:pandoracalonosce:Calorimetry 0.40912 0.900489 1.40029 0.9515 0.255774 30
produce:pandorapid:Chi2ParticleID 0.00160046 0.00243658 0.00440439 0.0023269 0.000708425 30
produce:pandoraShowercalo:ShowerCalorimetry 0.699256 2.13271 4.40338 2.14475 0.879335 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.606747 2.14281 4.89481 2.18638 0.957279 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0915853 0.198816 0.378878 0.18179 0.0695338 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0348732 0.0735356 0.13794 0.0670987 0.0277976 30
produce:emtrkmichelid:EmTrackMichelId 26.7387 43.3418 61.4308 42.709 9.35434 30
[art]:TriggerResults:TriggerResultInserter 2.8166e-05 5.23438e-05 0.000132916 4.68465e-05 1.96334e-05 30
end_path:out1:RootOutput 1.0135e-05 0.000396555 0.0114516 1.27605e-05 0.00205287 30
end_path:michelremoving:michelremoving 0.0297921 0.136357 0.730202 0.0553983 0.164293 30
end_path:michelremovinggnocchi:michelremoving 0.0270462 0.125831 0.730311 0.0529788 0.143069 30
end_path:calibanagnocchi:CalibAnaTree 0.235861 0.762233 1.54179 0.719403 0.324815 30
end_path:out1:RootOutput(write) 0.583445 1.03525 1.85859 1.00529 0.301476 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4930.32 MB
Peak resident set size usage (VmHWM): 2326.11 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_0794_dataflow5_datawriter_0_20240907T035156_reco_stage1_reco_stage2_20240926T154339_keepup.root
\tHists: np04hd_raw_run029107_0794_dataflow5_datawriter_0_20240907T035156_reco_stage1_reco_stage2_20240926T154339_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
