Jobsub ID 263306.96@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263306.96@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 10:01:13 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce01 |
| Last heartbeat | 2024-09-26 11:16:13 |
| From worker node | Hostname | ccwcondor0462 |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 10:14:51 |
| Input files | hd-protodune:np04hd_raw_run029081_0576_dataflow0_datawriter_0_20240905T014939.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:16:13 |
| Saved logs | justin-logs:263306.96-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
t list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 138488 at 26-Sep-2024 13:10:51 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 49834 traces, 49834 summary
Retagger: tagging trace set: gauss with 37248 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 37248 traces tagged "gauss"
FrameSaver: q=781171 n=677370 tag=gauss
wclsFrameSaver: saving 49834 traces tagged "wiener"
FrameSaver: q=823001 n=603437 tag=wiener
1223 hits in TPC 0
1576 hits in TPC 1
4114 hits in TPC 2
3338 hits in TPC 3
2278 hits in TPC 4
4401 hits in TPC 5
5889 hits in TPC 6
2640 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 2925 XUs and 1536 XVs -> 4 XUVs
C:0 T:2 21988 XUs and 35941 XVs -> 1382 XUVs
C:0 T:3 409 XUs and 439 XVs -> 8 XUVs
C:0 T:4 27 XUs and 51 XVs -> 3 XUVs
C:0 T:5 5981 XUs and 9195 XVs -> 507 XUVs
C:0 T:6 32461 XUs and 61452 XVs -> 990 XUVs
C:0 T:7 339 XUs and 613 XVs -> 6 XUVs
2900 XUVs total
1183 collection wire objects
2900 potential space points
Neighbour search...
224120 tests to find 104432 neighbours
Iterating with no regularization...
Begin: 7.25703e+06
0 7.01066e+06
1 6.99983e+06
2 6.9978e+06
Now with regularization...
Begin: 6.64023e+06
0 6.63928e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29081 subRun: 1 event: 138496 at 26-Sep-2024 13:11:59 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 83313 traces, 83313 summary
Retagger: tagging trace set: gauss with 64330 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 64330 traces tagged "gauss"
FrameSaver: q=1.40043e+06 n=1241822 tag=gauss
wclsFrameSaver: saving 83313 traces tagged "wiener"
FrameSaver: q=1.51809e+06 n=1134952 tag=wiener
5998 hits in TPC 0
7421 hits in TPC 1
10354 hits in TPC 2
5104 hits in TPC 3
4018 hits in TPC 4
4751 hits in TPC 5
7030 hits in TPC 6
3004 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 15728 XUs and 14087 XVs -> 364 XUVs
C:0 T:2 52783 XUs and 70426 XVs -> 4111 XUVs
C:0 T:3 203 XUs and 253 XVs -> 5 XUVs
C:0 T:4 213 XUs and 223 XVs -> 4 XUVs
C:0 T:5 21128 XUs and 12621 XVs -> 492 XUVs
C:0 T:6 30437 XUs and 39626 XVs -> 1818 XUVs
C:0 T:7 55 XUs and 97 XVs -> 3 XUVs
6797 XUVs total
2444 collection wire objects
6797 potential space points
Neighbour search...
533755 tests to find 260626 neighbours
Iterating with no regularization...
Begin: 9.60428e+06
0 8.66906e+06
1 8.60098e+06
2 8.59198e+06
3 8.58924e+06
Now with regularization...
Begin: 8.22309e+06
0 8.22142e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 138496 TrkackID 1 startx, y, z 151.649 584.74 461.5 wireno, peakt tpcno 18 5 5 dist 0min T, max_T5 2294
no of hits closeby 1 event 138496 TrkackID 7 startx, y, z -56.417 559.996 233.437 wireno, peakt tpcno 456 4452.45 6 dist 0min T, max_T4452.45 5251.07
no of hits closeby 0 event 138496 TrkackID 1 startx, y, z 151.649 584.74 461.5 wireno, peakt tpcno 18 5 5 dist 0min T, max_T5 2294
no of hits closeby 1 event 138496 TrkackID 7 startx, y, z -56.417 559.996 233.437 wireno, peakt tpcno 456 4452.45 6 dist 0min T, max_T4452.45 5251.07
26-Sep-2024 13:13:39 CEST Closed output file "np04hd_raw_run029081_0576_dataflow0_datawriter_0_20240905T014939_reco_stage1_reco_stage2_20240926T111339_keepup.root"
26-Sep-2024 13:13:39 CEST Closed input file "np04hd_raw_run029081_0576_dataflow0_datawriter_0_20240905T014939_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 55.0955 84.3169 123.116 83.9648 14.4755 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.00057285 0.00224217 0.00451068 0.00227897 0.000978693 30
produce:fembfilter:PDHDFEMBFilter 0.537217 0.575401 0.771175 0.553797 0.051633 30
produce:wclsdatahd:WireCellToolkit 34.5421 37.7373 42.1911 37.3964 1.78756 30
produce:gaushit:GausHitFinder 0.242941 0.61198 1.2081 0.566639 0.201066 30
produce:nhitsfilter:NumberOfHitsFilter 0.000161004 0.000358786 0.000693117 0.000352755 9.92401e-05 30
produce:reco3d:SpacePointSolver 0.158112 0.832857 2.34246 0.548821 0.635004 30
produce:hitpdune:DisambigFromSpacePoints 0.0897031 0.388327 0.960486 0.339866 0.203139 30
produce:pandora:StandardPandora 2.37244 10.8031 21.9182 9.42213 4.79075 30
produce:pandoraWriter:StandardPandora 0.0476944 0.137422 0.283526 0.125243 0.0508304 30
produce:pandoraTrack:LArPandoraTrackCreation 0.239063 0.983367 2.92119 0.841009 0.512137 30
produce:pandoraShower:LArPandoraShowerCreation 0.183461 1.09775 2.80611 1.01401 0.559761 30
produce:pandoracalo:Calorimetry 0.139793 0.465296 0.906649 0.459216 0.147805 30
produce:pandoracalonosce:Calorimetry 0.211504 0.426596 0.782008 0.386607 0.137111 30
produce:pandorapid:Chi2ParticleID 0.000950151 0.00150832 0.00293795 0.00141813 0.000358485 30
produce:pandoraShowercalo:ShowerCalorimetry 0.26284 0.984878 2.58784 0.961985 0.461118 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.260151 0.989742 2.48602 0.937488 0.473308 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0573437 0.127286 0.269371 0.115609 0.0470611 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0232526 0.0438105 0.0806169 0.0407993 0.0145469 30
produce:emtrkmichelid:EmTrackMichelId 13.1466 26.678 40.56 26.7357 6.25969 30
[art]:TriggerResults:TriggerResultInserter 1.7022e-05 3.09519e-05 0.000100079 2.35095e-05 1.98815e-05 30
end_path:out1:RootOutput 8.095e-06 1.14886e-05 2.4617e-05 9.934e-06 4.34864e-06 30
end_path:michelremoving:michelremoving 0.0177074 0.0701376 0.32607 0.0288625 0.0826296 30
end_path:michelremovinggnocchi:michelremoving 0.0156668 0.0660652 0.393422 0.0268291 0.0840077 30
end_path:calibanagnocchi:CalibAnaTree 0.188761 0.430151 0.901394 0.401628 0.193003 30
end_path:out1:RootOutput(write) 0.363811 0.756347 1.43782 0.728052 0.225967 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4932.12 MB
Peak resident set size usage (VmHWM): 2347.16 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0576_dataflow0_datawriter_0_20240905T014939_reco_stage1_reco_stage2_20240926T111339_keepup.root
\tHists: np04hd_raw_run029081_0576_dataflow0_datawriter_0_20240905T014939_reco_stage1_reco_stage2_20240926T111339_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
