Jobsub ID 263235.60@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263235.60@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3448 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:30:31 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 10:32:43 |
| From worker node | Hostname | ccwcondor0735.in2p3.fr |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:31:39 |
| Input files | hd-protodune:np04hd_raw_run029097_0129_dataflow1_datawriter_0_20240906T130236.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 10:32:43 |
| Saved logs | justin-logs:263235.60-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
Saver: saving 32021 traces tagged "gauss"
FrameSaver: q=691262 n=560122 tag=gauss
wclsFrameSaver: saving 44368 traces tagged "wiener"
FrameSaver: q=745868 n=499363 tag=wiener
1044 hits in TPC 0
1892 hits in TPC 1
2770 hits in TPC 2
1961 hits in TPC 3
2567 hits in TPC 4
5005 hits in TPC 5
5014 hits in TPC 6
2307 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 218 XUs and 491 XVs -> 21 XUVs
C:0 T:2 9328 XUs and 5705 XVs -> 135 XUVs
C:0 T:3 98 XUs and 56 XVs -> 3 XUVs
C:0 T:4 15 XUs and 27 XVs -> 0 XUVs
C:0 T:5 16860 XUs and 15495 XVs -> 467 XUVs
C:0 T:6 22544 XUs and 13799 XVs -> 402 XUVs
C:0 T:7 105 XUs and 123 XVs -> 32 XUVs
1060 XUVs total
641 collection wire objects
1060 potential space points
Neighbour search...
26176 tests to find 14154 neighbours
Iterating with no regularization...
Begin: 4.28539e+06
0 3.54361e+06
1 3.483e+06
2 3.47959e+06
Now with regularization...
Begin: 3.24634e+06
0 3.2449e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29097 subRun: 1 event: 31185 at 26-Sep-2024 12:29:35 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 48453 traces, 48453 summary
Retagger: tagging trace set: gauss with 36673 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 36673 traces tagged "gauss"
FrameSaver: q=793910 n=688369 tag=gauss
wclsFrameSaver: saving 48453 traces tagged "wiener"
FrameSaver: q=859469 n=627670 tag=wiener
1423 hits in TPC 0
1577 hits in TPC 1
6321 hits in TPC 2
3923 hits in TPC 3
2100 hits in TPC 4
3700 hits in TPC 5
5783 hits in TPC 6
2010 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 16 XUs and 5 XVs -> 0 XUVs
C:0 T:1 56 XUs and 28 XVs -> 1 XUVs
C:0 T:2 17012 XUs and 16441 XVs -> 623 XUVs
C:0 T:3 103 XUs and 193 XVs -> 10 XUVs
C:0 T:4 0 XUs and 0 XVs -> 0 XUVs
C:0 T:5 15750 XUs and 20815 XVs -> 2560 XUVs
C:0 T:6 11700 XUs and 16077 XVs -> 364 XUVs
C:0 T:7 734 XUs and 582 XVs -> 33 XUVs
3591 XUVs total
1190 collection wire objects
3591 potential space points
Neighbour search...
287487 tests to find 155672 neighbours
Iterating with no regularization...
Begin: 7.12832e+06
0 6.27206e+06
1 6.20673e+06
2 6.19718e+06
3 6.1937e+06
Now with regularization...
Begin: 5.71944e+06
0 5.71627e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29097 subRun: 1 event: 31193 at 26-Sep-2024 12:30:42 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 46789 traces, 46789 summary
Retagger: tagging trace set: gauss with 37392 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 37392 traces tagged "gauss"
FrameSaver: q=819786 n=745554 tag=gauss
wclsFrameSaver: saving 46789 traces tagged "wiener"
FrameSaver: q=859298 n=679961 tag=wiener
971 hits in TPC 0
802 hits in TPC 1
8600 hits in TPC 2
3420 hits in TPC 3
2726 hits in TPC 4
4496 hits in TPC 5
2836 hits in TPC 6
1630 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 18 XUs and 45 XVs -> 0 XUVs
C:0 T:2 28234 XUs and 22629 XVs -> 548 XUVs
C:0 T:3 1593 XUs and 2288 XVs -> 64 XUVs
C:0 T:4 94 XUs and 84 XVs -> 10 XUVs
C:0 T:5 8882 XUs and 5666 XVs -> 144 XUVs
C:0 T:6 2955 XUs and 2759 XVs -> 242 XUVs
C:0 T:7 98 XUs and 121 XVs -> 11 XUVs
1019 XUVs total
657 collection wire objects
1019 potential space points
Neighbour search...
11709 tests to find 6880 neighbours
Iterating with no regularization...
Begin: 2.74832e+06
0 2.60764e+06
1 2.60303e+06
2 2.6025e+06
Now with regularization...
Begin: 2.44598e+06
0 2.44576e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:31:57 CEST Closed output file "np04hd_raw_run029097_0129_dataflow1_datawriter_0_20240906T130236_reco_stage1_reco_stage2_20240926T103157_keepup.root"
26-Sep-2024 12:31:57 CEST Closed input file "np04hd_raw_run029097_0129_dataflow1_datawriter_0_20240906T130236_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 60.9393 85.422 126.819 84.5087 15.6985 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000345932 0.00264893 0.0103853 0.00231744 0.0018991 30
produce:fembfilter:PDHDFEMBFilter 0.536256 0.624896 0.811422 0.622569 0.0693363 30
produce:wclsdatahd:WireCellToolkit 35.8657 40.3678 47.9807 39.9096 3.38362 30
produce:gaushit:GausHitFinder 0.303707 0.585632 0.928404 0.560275 0.166935 30
produce:nhitsfilter:NumberOfHitsFilter 0.000183515 0.000382632 0.000739332 0.000380251 0.000117508 30
produce:reco3d:SpacePointSolver 0.187553 0.759418 2.09573 0.621502 0.490943 30
produce:hitpdune:DisambigFromSpacePoints 0.12764 0.442374 1.10894 0.420661 0.20459 30
produce:pandora:StandardPandora 3.97879 10.1979 24.1273 9.12724 4.56684 30
produce:pandoraWriter:StandardPandora 0.0615596 0.130455 0.22621 0.128171 0.042884 30
produce:pandoraTrack:LArPandoraTrackCreation 0.265209 0.94685 1.88006 0.894204 0.373665 30
produce:pandoraShower:LArPandoraShowerCreation 0.303207 1.13732 2.54224 1.04924 0.523465 30
produce:pandoracalo:Calorimetry 0.163761 0.516335 0.897087 0.505021 0.181179 30
produce:pandoracalonosce:Calorimetry 0.133093 0.449436 0.905063 0.43058 0.168035 30
produce:pandorapid:Chi2ParticleID 0.000878454 0.00160982 0.00257338 0.00145051 0.000453006 30
produce:pandoraShowercalo:ShowerCalorimetry 0.267782 1.01407 2.33481 0.930094 0.472628 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.260762 0.972828 2.59021 0.880182 0.470495 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0410542 0.12402 0.246415 0.118759 0.0421177 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0170886 0.0416656 0.0973014 0.0393389 0.0148748 30
produce:emtrkmichelid:EmTrackMichelId 15.8696 25.7891 39.6614 26.5455 6.37549 30
[art]:TriggerResults:TriggerResultInserter 1.6932e-05 3.89717e-05 0.000119736 2.87995e-05 2.46686e-05 30
end_path:out1:RootOutput 6.793e-06 1.0613e-05 3.17e-05 8.386e-06 5.70188e-06 30
end_path:michelremoving:michelremoving 0.0143948 0.0684375 0.47158 0.0292869 0.0901922 30
end_path:michelremovinggnocchi:michelremoving 0.0131343 0.0600903 0.358319 0.0246181 0.072477 30
end_path:calibanagnocchi:CalibAnaTree 0.121557 0.362405 0.743674 0.361498 0.139133 30
end_path:out1:RootOutput(write) 0.39974 0.710995 1.19459 0.73659 0.169425 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4922.14 MB
Peak resident set size usage (VmHWM): 2345.59 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0129_dataflow1_datawriter_0_20240906T130236_reco_stage1_reco_stage2_20240926T103157_keepup.root
\tHists: np04hd_raw_run029097_0129_dataflow1_datawriter_0_20240906T130236_reco_stage1_reco_stage2_20240926T103157_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
