Jobsub ID 263765.53@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263765.53@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3451 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 13:24:52 |
| Site | ES_PIC |
| Entry | DUNE_T1_ES_PIC_ce14-multicore |
| Last heartbeat | 2024-09-26 14:43:14 |
| From worker node | Hostname | tds222.pic.es |
| cpuinfo | Intel(R) Xeon(R) CPU E5-2640 v3 @ 2.60GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 13:26:38 |
| Input files | hd-protodune:np04hd_raw_run029107_2218_dataflow1_datawriter_0_20240908T035156.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:43:14 |
| Saved logs | justin-logs:263765.53-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
objects
1261 potential space points
Neighbour search...
20289 tests to find 11270 neighbours
Iterating with no regularization...
Begin: 3.74925e+06
0 3.50481e+06
1 3.48871e+06
2 3.48603e+06
Now with regularization...
Begin: 3.22737e+06
0 3.22669e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 532561 TrkackID 1 startx, y, z 123.095 596.974 354.416 wireno, peakt tpcno 114 5484.97 6 dist 375.614min T, max_T3021.92 5510.67
no of hits closeby 0 event 532561 TrkackID 1 startx, y, z 123.095 596.974 354.416 wireno, peakt tpcno 114 5484.97 6 dist 375.614min T, max_T3021.92 5510.67
Begin processing the 29th record. run: 29107 subRun: 1 event: 532577 at 26-Sep-2024 16:40:12 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 65368 traces, 65368 summary
Retagger: tagging trace set: gauss with 48166 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48166 traces tagged "gauss"
FrameSaver: q=963829 n=896323 tag=gauss
wclsFrameSaver: saving 65368 traces tagged "wiener"
FrameSaver: q=1.04657e+06 n=811959 tag=wiener
4352 hits in TPC 0
4624 hits in TPC 1
7446 hits in TPC 2
3093 hits in TPC 3
3277 hits in TPC 4
5929 hits in TPC 5
5460 hits in TPC 6
1731 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1860 XUs and 1617 XVs -> 6 XUVs
C:0 T:2 36165 XUs and 32697 XVs -> 488 XUVs
C:0 T:3 68 XUs and 71 XVs -> 3 XUVs
C:0 T:4 414 XUs and 537 XVs -> 83 XUVs
C:0 T:5 16204 XUs and 25430 XVs -> 393 XUVs
C:0 T:6 47306 XUs and 65381 XVs -> 2316 XUVs
C:0 T:7 2 XUs and 6 XVs -> 0 XUVs
3289 XUVs total
1118 collection wire objects
3289 potential space points
Neighbour search...
110839 tests to find 53818 neighbours
Iterating with no regularization...
Begin: 8.98807e+06
0 6.89631e+06
1 6.78972e+06
2 6.77928e+06
3 6.77675e+06
Now with regularization...
Begin: 6.57469e+06
0 6.57358e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29107 subRun: 1 event: 532585 at 26-Sep-2024 16:41:33 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66271 traces, 66271 summary
Retagger: tagging trace set: gauss with 48834 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48834 traces tagged "gauss"
FrameSaver: q=1.11814e+06 n=874715 tag=gauss
wclsFrameSaver: saving 66271 traces tagged "wiener"
FrameSaver: q=1.20071e+06 n=789421 tag=wiener
1278 hits in TPC 0
1400 hits in TPC 1
7228 hits in TPC 2
3774 hits in TPC 3
3140 hits in TPC 4
5054 hits in TPC 5
5785 hits in TPC 6
5841 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 104 XUs and 17 XVs -> 0 XUVs
C:0 T:2 25052 XUs and 32850 XVs -> 1179 XUVs
C:0 T:3 31 XUs and 41 XVs -> 1 XUVs
C:0 T:4 35 XUs and 68 XVs -> 6 XUVs
C:0 T:5 35472 XUs and 16220 XVs -> 493 XUVs
C:0 T:6 28895 XUs and 37259 XVs -> 1065 XUVs
C:0 T:7 189 XUs and 248 XVs -> 14 XUVs
2758 XUVs total
1375 collection wire objects
2758 potential space points
Neighbour search...
102546 tests to find 58716 neighbours
Iterating with no regularization...
Begin: 8.89126e+06
0 8.54642e+06
1 8.52444e+06
2 8.52098e+06
Now with regularization...
Begin: 8.05256e+06
0 8.05112e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 532585 TrkackID 5 startx, y, z 62.4882 544.165 440.778 wireno, peakt tpcno 223 303.119 5 dist 0min T, max_T3.36208 833.538
no of hits closeby 1 event 532585 TrkackID 5 startx, y, z 62.4882 544.165 440.778 wireno, peakt tpcno 223 303.119 5 dist 0min T, max_T3.36208 833.538
26-Sep-2024 16:42:54 CEST Closed output file "np04hd_raw_run029107_2218_dataflow1_datawriter_0_20240908T035156_reco_stage1_reco_stage2_20240926T144254_keepup.root"
26-Sep-2024 16:42:54 CEST Closed input file "np04hd_raw_run029107_2218_dataflow1_datawriter_0_20240908T035156_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 61.4302 79.8119 109.386 79.9606 9.92616 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000268567 0.00127699 0.00212885 0.00129289 0.000556014 30
produce:fembfilter:PDHDFEMBFilter 0.522588 0.534004 0.560404 0.531866 0.00837686 30
produce:wclsdatahd:WireCellToolkit 36.4786 39.1115 42.0318 38.7404 1.33677 30
produce:gaushit:GausHitFinder 0.411737 0.628618 0.967675 0.619086 0.115583 30
produce:nhitsfilter:NumberOfHitsFilter 0.000168117 0.000293793 0.00043876 0.000296832 7.47775e-05 30
produce:reco3d:SpacePointSolver 0.172778 0.751207 2.24485 0.720786 0.495106 30
produce:hitpdune:DisambigFromSpacePoints 0.173173 0.458362 1.23061 0.375365 0.237678 30
produce:pandora:StandardPandora 5.4941 11.0828 23.2218 10.6008 3.63038 30
produce:pandoraWriter:StandardPandora 0.0790661 0.122358 0.182807 0.120428 0.0227279 30
produce:pandoraTrack:LArPandoraTrackCreation 0.481148 1.02127 1.70621 0.942552 0.294132 30
produce:pandoraShower:LArPandoraShowerCreation 0.320808 1.21273 2.45799 1.11276 0.45886 30
produce:pandoracalo:Calorimetry 0.252983 0.520829 0.815412 0.522017 0.132128 30
produce:pandoracalonosce:Calorimetry 0.230279 0.471263 0.724202 0.469013 0.119608 30
produce:pandorapid:Chi2ParticleID 0.00108684 0.00149163 0.00254995 0.00145346 0.000254413 30
produce:pandoraShowercalo:ShowerCalorimetry 0.271982 1.12346 2.15439 1.08626 0.414901 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.266724 1.09976 2.10456 1.05783 0.406592 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.063248 0.109361 0.169856 0.107269 0.0235554 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0215513 0.038643 0.0572776 0.038141 0.00782653 30
produce:emtrkmichelid:EmTrackMichelId 13.0943 20.2511 29.8453 19.8151 3.73338 30
[art]:TriggerResults:TriggerResultInserter 1.4913e-05 2.29571e-05 7.1696e-05 2.028e-05 1.17353e-05 30
end_path:out1:RootOutput 3.34e-06 6.97473e-06 2.0871e-05 6.446e-06 3.03543e-06 30
end_path:michelremoving:michelremoving 0.0157704 0.0754897 0.269193 0.0339316 0.0668536 30
end_path:michelremovinggnocchi:michelremoving 0.0147604 0.0738522 0.267777 0.0328369 0.0669046 30
end_path:calibanagnocchi:CalibAnaTree 0.183249 0.428904 0.738628 0.418823 0.118552 30
end_path:out1:RootOutput(write) 0.384674 0.625891 0.93875 0.620673 0.117821 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4950 MB
Peak resident set size usage (VmHWM): 2592.98 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029107_2218_dataflow1_datawriter_0_20240908T035156_reco_stage1_reco_stage2_20240926T144254_keepup.root
\tHists: np04hd_raw_run029107_2218_dataflow1_datawriter_0_20240908T035156_reco_stage1_reco_stage2_20240926T144254_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
