Jobsub ID 263283.141@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263283.141@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:56:54 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:26:51 |
| From worker node | Hostname | ccwcondor0488 |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:59:19 |
| Input files | hd-protodune:np04hd_raw_run029081_0938_dataflow1_datawriter_0_20240905T073340.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:26:51 |
| Saved logs | justin-logs:263283.141-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
:6 47994 XUs and 50726 XVs -> 1739 XUVs
C:0 T:7 70 XUs and 113 XVs -> 1 XUVs
4888 XUVs total
2070 collection wire objects
4888 potential space points
Neighbour search...
197726 tests to find 113860 neighbours
Iterating with no regularization...
Begin: 1.02418e+07
0 9.15034e+06
1 9.08016e+06
2 9.06938e+06
3 9.06615e+06
Now with regularization...
Begin: 8.42184e+06
0 8.41942e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 225345 at 26-Sep-2024 13:07:34 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 44696 traces, 44696 summary
Retagger: tagging trace set: gauss with 33205 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 33205 traces tagged "gauss"
FrameSaver: q=684803 n=612713 tag=gauss
wclsFrameSaver: saving 44696 traces tagged "wiener"
FrameSaver: q=740514 n=557221 tag=wiener
2918 hits in TPC 0
3445 hits in TPC 1
5057 hits in TPC 2
3026 hits in TPC 3
581 hits in TPC 4
1406 hits in TPC 5
4569 hits in TPC 6
2340 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 4396 XUs and 2074 XVs -> 23 XUVs
C:0 T:2 17966 XUs and 27365 XVs -> 1338 XUVs
C:0 T:3 278 XUs and 315 XVs -> 18 XUVs
C:0 T:4 49 XUs and 45 XVs -> 3 XUVs
C:0 T:5 984 XUs and 3800 XVs -> 3 XUVs
C:0 T:6 12327 XUs and 46436 XVs -> 629 XUVs
C:0 T:7 127 XUs and 160 XVs -> 5 XUVs
2019 XUVs total
981 collection wire objects
2019 potential space points
Neighbour search...
62229 tests to find 33156 neighbours
Iterating with no regularization...
Begin: 4.22899e+06
0 3.77094e+06
1 3.74033e+06
2 3.738e+06
Now with regularization...
Begin: 3.4488e+06
0 3.44777e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 225353 at 26-Sep-2024 13:08:44 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 47011 traces, 47011 summary
Retagger: tagging trace set: gauss with 36429 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 36429 traces tagged "gauss"
FrameSaver: q=772634 n=710594 tag=gauss
wclsFrameSaver: saving 47011 traces tagged "wiener"
FrameSaver: q=812249 n=643076 tag=wiener
4007 hits in TPC 0
3758 hits in TPC 1
4278 hits in TPC 2
2700 hits in TPC 3
2233 hits in TPC 4
3368 hits in TPC 5
1925 hits in TPC 6
2372 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 188 XUs and 212 XVs -> 12 XUVs
C:0 T:2 6893 XUs and 10643 XVs -> 791 XUVs
C:0 T:3 35 XUs and 63 XVs -> 4 XUVs
C:0 T:4 8 XUs and 12 XVs -> 0 XUVs
C:0 T:5 6641 XUs and 5125 XVs -> 192 XUVs
C:0 T:6 3187 XUs and 3503 XVs -> 296 XUVs
C:0 T:7 4 XUs and 4 XVs -> 0 XUVs
1295 XUVs total
840 collection wire objects
1295 potential space points
Neighbour search...
20197 tests to find 11326 neighbours
Iterating with no regularization...
Begin: 2.30205e+06
0 2.10443e+06
1 2.08955e+06
2 2.08803e+06
Now with regularization...
Begin: 1.83269e+06
0 1.83242e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 2 event 225353 TrkackID 2 startx, y, z -63.521 427.088 11.9562 wireno, peakt tpcno 167 6.03955 6 dist 0min T, max_T2.71513 1516.25
no of hits closeby 2 event 225353 TrkackID 2 startx, y, z -63.521 427.088 11.9562 wireno, peakt tpcno 167 6.03955 6 dist 0min T, max_T2.71513 1516.25
26-Sep-2024 13:10:03 CEST Closed output file "np04hd_raw_run029081_0938_dataflow1_datawriter_0_20240905T073340_reco_stage1_reco_stage2_20240926T111003_keepup.root"
26-Sep-2024 13:10:03 CEST Closed input file "np04hd_raw_run029081_0938_dataflow1_datawriter_0_20240905T073340_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 61.4549 99.8733 143.982 97.8684 18.7226 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000413771 0.00271321 0.00578888 0.00282478 0.00124448 30
produce:fembfilter:PDHDFEMBFilter 0.626684 0.748258 1.00531 0.704293 0.103809 30
produce:wclsdatahd:WireCellToolkit 40.1917 46.9797 53.2035 45.5937 4.05562 30
produce:gaushit:GausHitFinder 0.272859 0.762668 1.3675 0.737013 0.23243 30
produce:nhitsfilter:NumberOfHitsFilter 0.000256094 0.00475893 0.126394 0.000440927 0.0225958 30
produce:reco3d:SpacePointSolver 0.163228 0.929568 2.9106 0.817855 0.602948 30
produce:hitpdune:DisambigFromSpacePoints 0.128722 0.499849 1.03235 0.469618 0.226904 30
produce:pandora:StandardPandora 3.28659 11.8644 23.3631 11.4532 5.13495 30
produce:pandoraWriter:StandardPandora 0.055352 0.157877 0.275606 0.15439 0.0532701 30
produce:pandoraTrack:LArPandoraTrackCreation 0.266693 1.15657 2.74338 1.10567 0.526899 30
produce:pandoraShower:LArPandoraShowerCreation 0.21841 1.24843 3.03548 1.1833 0.590513 30
produce:pandoracalo:Calorimetry 0.168787 0.584198 1.09122 0.601581 0.199098 30
produce:pandoracalonosce:Calorimetry 0.140423 0.542112 1.11343 0.569464 0.210929 30
produce:pandorapid:Chi2ParticleID 0.000884751 0.00195326 0.00375982 0.00175491 0.000640593 30
produce:pandoraShowercalo:ShowerCalorimetry 0.209555 1.19079 2.36997 1.1729 0.510407 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.207799 1.14257 2.60126 1.09543 0.521042 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0615402 0.148495 0.261187 0.12815 0.0518352 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0206 0.0496261 0.0793076 0.0470646 0.0151143 30
produce:emtrkmichelid:EmTrackMichelId 14.1086 30.3152 47.6632 29.7424 7.63717 30
[art]:TriggerResults:TriggerResultInserter 2.1751e-05 4.40198e-05 0.000106862 3.90385e-05 1.83901e-05 30
end_path:out1:RootOutput 8.336e-06 1.12769e-05 3.5277e-05 1.0334e-05 4.82067e-06 30
end_path:michelremoving:michelremoving 0.0169641 0.0888155 0.243224 0.0413661 0.0720921 30
end_path:michelremovinggnocchi:michelremoving 0.0180162 0.0822969 0.231849 0.0390593 0.0682725 30
end_path:calibanagnocchi:CalibAnaTree 0.134443 0.40359 0.809882 0.402717 0.155141 30
end_path:out1:RootOutput(write) 0.367863 0.81459 1.24739 0.799501 0.209531 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4915.9 MB
Peak resident set size usage (VmHWM): 2339.3 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0938_dataflow1_datawriter_0_20240905T073340_reco_stage1_reco_stage2_20240926T111003_keepup.root
\tHists: np04hd_raw_run029081_0938_dataflow1_datawriter_0_20240905T073340_reco_stage1_reco_stage2_20240926T111003_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
