Jobsub ID 263315.145@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263315.145@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 10:03:50 |
| Site | IT_CNAF |
| Entry | CMSHTPC_T1_IT_CNAF_condor_ce02 |
| Last heartbeat | 2024-09-26 12:04:11 |
| From worker node | Hostname | cn-leo-031.cr.cnaf.infn.it |
| cpuinfo | Intel(R) Xeon(R) Platinum 8480+ |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 216000 (60 hours) |
| Inner Apptainer? | True |
| Job state | outputting_failed |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:28:10 |
| Input files | hd-protodune:np04hd_raw_run029081_1045_dataflow0_datawriter_0_20240905T093727.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 12:04:11 |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
at 26-Sep-2024 13:34:35 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 45437 traces, 45437 summary
Retagger: tagging trace set: gauss with 33808 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 33808 traces tagged "gauss"
FrameSaver: q=755519 n=628493 tag=gauss
wclsFrameSaver: saving 45437 traces tagged "wiener"
FrameSaver: q=801395 n=564427 tag=wiener
917 hits in TPC 0
991 hits in TPC 1
3432 hits in TPC 2
2628 hits in TPC 3
3298 hits in TPC 4
4227 hits in TPC 5
4922 hits in TPC 6
2788 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 10 XUs and 6 XVs -> 0 XUVs
C:0 T:2 6292 XUs and 7749 XVs -> 250 XUVs
C:0 T:3 9 XUs and 19 XVs -> 1 XUVs
C:0 T:4 27 XUs and 59 XVs -> 1 XUVs
C:0 T:5 14544 XUs and 14698 XVs -> 975 XUVs
C:0 T:6 11157 XUs and 10073 XVs -> 789 XUVs
C:0 T:7 345 XUs and 471 XVs -> 24 XUVs
2040 XUVs total
802 collection wire objects
2040 potential space points
Neighbour search...
120464 tests to find 71378 neighbours
Iterating with no regularization...
Begin: 5.63369e+06
0 5.13202e+06
1 5.09778e+06
2 5.09281e+06
Now with regularization...
Begin: 4.7224e+06
0 4.71968e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 251072 at 26-Sep-2024 13:36:03 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 63789 traces, 63789 summary
Retagger: tagging trace set: gauss with 47153 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47153 traces tagged "gauss"
FrameSaver: q=985112 n=844286 tag=gauss
wclsFrameSaver: saving 63789 traces tagged "wiener"
FrameSaver: q=1.06058e+06 n=754649 tag=wiener
4037 hits in TPC 0
3904 hits in TPC 1
5604 hits in TPC 2
3041 hits in TPC 3
2446 hits in TPC 4
6135 hits in TPC 5
4950 hits in TPC 6
2243 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 652 XUs and 329 XVs -> 7 XUVs
C:0 T:2 33398 XUs and 34476 XVs -> 859 XUVs
C:0 T:3 34 XUs and 40 XVs -> 1 XUVs
C:0 T:4 113 XUs and 166 XVs -> 6 XUVs
C:0 T:5 23928 XUs and 19536 XVs -> 1598 XUVs
C:0 T:6 21447 XUs and 55368 XVs -> 701 XUVs
C:0 T:7 90 XUs and 113 XVs -> 8 XUVs
3180 XUVs total
1115 collection wire objects
3180 potential space points
Neighbour search...
135250 tests to find 83112 neighbours
Iterating with no regularization...
Begin: 9.68672e+06
0 8.55966e+06
1 8.4583e+06
2 8.44554e+06
3 8.44137e+06
Now with regularization...
Begin: 8.0361e+06
0 8.03348e+06
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00800323 0.137535 0.0576935 0.0096817 0.02491 0.0306091 0.024498 0.0105515 0.00912476 0.0143127 0.00523376
----view 1: 0.00023818 0.000238419 0.00023818 0.000238419 0.00023818 0.000238657 0.00023818 0.00023818 0.000238419 0.00023818 0.000238419
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.137535 0.0576935 0.0096817 0.02491 0.0306091 0.024498 0.0105515 0.00912476 0.0143127 0.00523376 0.0181808
----view 1: 0.000238419 0.00023818 0.000238419 0.00023818 0.000238657 0.00023818 0.00023818 0.000238419 0.00023818 0.000238419 0.000238419
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0576935 0.0096817 0.02491 0.0306091 0.024498 0.0105515 0.00912476 0.0143127 0.00523376 0.0181808 0.0145645
----view 1: 0.00023818 0.000238419 0.00023818 0.000238657 0.00023818 0.00023818 0.000238419 0.00023818 0.000238419 0.000238419 0.000238419
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 251072 TrkackID 18 startx, y, z 54.6428 593.901 197.87 wireno, peakt tpcno 365 3746.42 2 dist 65.9062min T, max_T3083.28 3773.59
no of hits closeby 0 event 251072 TrkackID 18 startx, y, z 54.6428 593.901 197.87 wireno, peakt tpcno 365 3746.42 2 dist 65.9062min T, max_T3083.28 3773.59
26-Sep-2024 13:37:41 CEST Closed output file "np04hd_raw_run029081_1045_dataflow0_datawriter_0_20240905T093727_reco_stage1_reco_stage2_20240926T113741_keepup.root"
26-Sep-2024 13:37:41 CEST Closed input file "np04hd_raw_run029081_1045_dataflow0_datawriter_0_20240905T093727_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 74.7212 99.13 137.526 96.3143 14.0492 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000512619 0.00221478 0.00548977 0.0019467 0.00149931 30
produce:fembfilter:PDHDFEMBFilter 0.672243 0.713823 0.769221 0.709138 0.0225728 30
produce:wclsdatahd:WireCellToolkit 55.7152 59.7562 63.5118 60.1131 2.09388 30
produce:gaushit:GausHitFinder 0.345483 0.547433 0.874814 0.541955 0.12534 30
produce:nhitsfilter:NumberOfHitsFilter 0.000335557 0.000498232 0.000843141 0.000476346 0.000123646 30
produce:reco3d:SpacePointSolver 0.172807 0.969599 4.92853 0.465574 1.00003 30
produce:hitpdune:DisambigFromSpacePoints 0.14614 0.313966 0.818427 0.270173 0.15031 30
produce:pandora:StandardPandora 3.97053 9.8459 25.4034 8.93712 4.88128 30
produce:pandoraWriter:StandardPandora 0.0877873 0.149171 0.213423 0.151833 0.0351595 30
produce:pandoraTrack:LArPandoraTrackCreation 0.662633 1.98129 3.87982 2.00573 0.692676 30
produce:pandoraShower:LArPandoraShowerCreation 0.72136 2.54987 6.59706 2.49042 1.10092 30
produce:pandoracalo:Calorimetry 0.291116 0.821129 1.76584 0.829882 0.291401 30
produce:pandoracalonosce:Calorimetry 0.263608 0.767891 1.70754 0.778515 0.278512 30
produce:pandorapid:Chi2ParticleID 0.00108412 0.00154797 0.00284272 0.0015321 0.0003351 30
produce:pandoraShowercalo:ShowerCalorimetry 0.573402 2.03995 6.08646 1.97932 0.993476 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.556849 1.97752 5.92973 1.95873 0.95531 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0526635 0.0923385 0.143919 0.0951883 0.0216347 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0223687 0.0402675 0.0584956 0.041742 0.00902918 30
produce:emtrkmichelid:EmTrackMichelId 8.82648 14.9156 25.6848 14.4422 3.71032 30
[art]:TriggerResults:TriggerResultInserter 3.6743e-05 6.22082e-05 9.0422e-05 6.195e-05 1.32839e-05 30
end_path:out1:RootOutput 1.2297e-05 1.9124e-05 2.8586e-05 1.8895e-05 3.82502e-06 30
end_path:michelremoving:michelremoving 0.0209474 0.0740195 0.155771 0.0584188 0.0366998 30
end_path:michelremovinggnocchi:michelremoving 0.0191396 0.0718378 0.155593 0.0610331 0.036233 30
end_path:calibanagnocchi:CalibAnaTree 0.153753 0.572997 1.2781 0.544152 0.207177 30
end_path:out1:RootOutput(write) 0.374658 0.59923 0.968098 0.595784 0.145774 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5772.32 MB
Peak resident set size usage (VmHWM): 2603.11 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_1045_dataflow0_datawriter_0_20240905T093727_reco_stage1_reco_stage2_20240926T113741_keepup.root
\tHists: np04hd_raw_run029081_1045_dataflow0_datawriter_0_20240905T093727_reco_stage1_reco_stage2_20240926T113741_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
