Jobsub ID 263283.143@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263283.143@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:56:54 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 11:10:48 |
| From worker node | Hostname | ccwcondor0553 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:58:49 |
| Input files | hd-protodune:np04hd_raw_run029081_1087_dataflow1_datawriter_0_20240905T102629.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:10:48 |
| Saved logs | justin-logs:263283.143-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
search...
231783 tests to find 138900 neighbours
Iterating with no regularization...
Begin: 8.2742e+06
0 7.67232e+06
1 7.62792e+06
2 7.62089e+06
Now with regularization...
Begin: 7.18313e+06
0 7.18084e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 261105 at 26-Sep-2024 12:58:13 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 102497 traces, 102497 summary
Retagger: tagging trace set: gauss with 74911 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 74911 traces tagged "gauss"
FrameSaver: q=1.5719e+06 n=1340629 tag=gauss
wclsFrameSaver: saving 102497 traces tagged "wiener"
FrameSaver: q=1.71066e+06 n=1206286 tag=wiener
5293 hits in TPC 0
5901 hits in TPC 1
8072 hits in TPC 2
5141 hits in TPC 3
6305 hits in TPC 4
9910 hits in TPC 5
7447 hits in TPC 6
4132 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 7622 XUs and 8156 XVs -> 577 XUVs
C:0 T:2 55765 XUs and 59927 XVs -> 3597 XUVs
C:0 T:3 306 XUs and 198 XVs -> 7 XUVs
C:0 T:4 279 XUs and 348 XVs -> 8 XUVs
C:0 T:5 49384 XUs and 76378 XVs -> 2323 XUVs
C:0 T:6 43549 XUs and 38332 XVs -> 1267 XUVs
C:0 T:7 69 XUs and 60 XVs -> 2 XUVs
7781 XUVs total
2338 collection wire objects
7781 potential space points
Neighbour search...
781901 tests to find 376874 neighbours
Iterating with no regularization...
Begin: 1.21425e+07
0 1.10727e+07
1 1.09974e+07
2 1.09877e+07
Now with regularization...
Begin: 1.0498e+07
0 1.04942e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 261105 TrkackID 0 startx, y, z 139.983 601.037 124.897 wireno, peakt tpcno 477 728.789 5 dist 0min T, max_T725.411 3627
no of hits closeby 1 event 261105 TrkackID 0 startx, y, z 139.983 601.037 124.897 wireno, peakt tpcno 477 728.789 5 dist 0min T, max_T725.411 3627
Begin processing the 30th record. run: 29081 subRun: 1 event: 261113 at 26-Sep-2024 13:00:06 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 59451 traces, 59451 summary
Retagger: tagging trace set: gauss with 45089 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 45089 traces tagged "gauss"
FrameSaver: q=1.00974e+06 n=854448 tag=gauss
wclsFrameSaver: saving 59451 traces tagged "wiener"
FrameSaver: q=1.07828e+06 n=777047 tag=wiener
2222 hits in TPC 0
3500 hits in TPC 1
4292 hits in TPC 2
3832 hits in TPC 3
3589 hits in TPC 4
6536 hits in TPC 5
4171 hits in TPC 6
2989 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 1012 XUs and 729 XVs -> 37 XUVs
C:0 T:2 13340 XUs and 9028 XVs -> 554 XUVs
C:0 T:3 42 XUs and 67 XVs -> 1 XUVs
C:0 T:4 183 XUs and 314 XVs -> 25 XUVs
C:0 T:5 26245 XUs and 33031 XVs -> 1572 XUVs
C:0 T:6 6962 XUs and 8851 XVs -> 507 XUVs
C:0 T:7 347 XUs and 470 XVs -> 3 XUVs
2699 XUVs total
1182 collection wire objects
2699 potential space points
Neighbour search...
93073 tests to find 57192 neighbours
Iterating with no regularization...
Begin: 1.10976e+07
0 1.04997e+07
1 1.04503e+07
2 1.04413e+07
Now with regularization...
Begin: 9.69812e+06
0 9.69422e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 261113 TrkackID 12 startx, y, z 128.124 599.923 198.841 wireno, peakt tpcno 462 5083.31 2 dist 64.509min T, max_T3466.47 5099.75
no of hits closeby 0 event 261113 TrkackID 12 startx, y, z 128.124 599.923 198.841 wireno, peakt tpcno 462 5083.31 2 dist 64.509min T, max_T3466.47 5099.75
26-Sep-2024 13:01:18 CEST Closed output file "np04hd_raw_run029081_1087_dataflow1_datawriter_0_20240905T102629_reco_stage1_reco_stage2_20240926T110118_keepup.root"
26-Sep-2024 13:01:18 CEST Closed input file "np04hd_raw_run029081_1087_dataflow1_datawriter_0_20240905T102629_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 53.5039 89.7128 126.565 89.9735 19.3365 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000437529 0.00287994 0.0054177 0.00306154 0.00134986 30
produce:fembfilter:PDHDFEMBFilter 0.494949 0.64208 0.781799 0.683004 0.107778 30
produce:wclsdatahd:WireCellToolkit 30.2763 37.6313 44.5749 39.3803 5.3426 30
produce:gaushit:GausHitFinder 0.310571 0.664971 1.07708 0.60674 0.2138 30
produce:nhitsfilter:NumberOfHitsFilter 0.000207668 0.000343201 0.000779318 0.000314704 0.000102835 30
produce:reco3d:SpacePointSolver 0.199758 0.963809 3.20438 0.848976 0.625307 30
produce:hitpdune:DisambigFromSpacePoints 0.137746 0.524143 1.45993 0.424667 0.283071 30
produce:pandora:StandardPandora 3.87433 11.8561 24.2672 10.9297 5.12568 30
produce:pandoraWriter:StandardPandora 0.0811648 0.159322 0.287572 0.151405 0.049533 30
produce:pandoraTrack:LArPandoraTrackCreation 0.632271 1.87167 3.47997 1.84004 0.767021 30
produce:pandoraShower:LArPandoraShowerCreation 0.44691 2.33993 5.15887 2.35596 1.36335 30
produce:pandoracalo:Calorimetry 0.269886 0.876138 2.20077 0.892721 0.417341 30
produce:pandoracalonosce:Calorimetry 0.207815 0.800877 1.71345 0.756453 0.36257 30
produce:pandorapid:Chi2ParticleID 0.00110353 0.00209609 0.00459064 0.00185432 0.000711708 30
produce:pandoraShowercalo:ShowerCalorimetry 0.368903 1.9468 4.54667 1.8143 1.10173 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.371358 1.84712 4.27774 1.67036 1.0157 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0561831 0.136346 0.281101 0.130969 0.0527149 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0235294 0.0496463 0.0962422 0.0476762 0.0180468 30
produce:emtrkmichelid:EmTrackMichelId 14.7233 25.6292 35.8724 25.1609 6.10034 30
[art]:TriggerResults:TriggerResultInserter 2.5056e-05 6.42848e-05 0.00010688 6.62585e-05 1.84132e-05 30
end_path:out1:RootOutput 9.427e-06 2.02384e-05 6.7256e-05 1.85395e-05 9.98826e-06 30
end_path:michelremoving:michelremoving 0.0191257 0.102199 0.39514 0.0575714 0.0991999 30
end_path:michelremovinggnocchi:michelremoving 0.0182425 0.099329 0.396233 0.050373 0.0971668 30
end_path:calibanagnocchi:CalibAnaTree 0.210031 0.599989 1.18107 0.553681 0.283933 30
end_path:out1:RootOutput(write) 0.357581 0.744168 1.13281 0.777033 0.204273 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4900.25 MB
Peak resident set size usage (VmHWM): 2331.43 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_1087_dataflow1_datawriter_0_20240905T102629_reco_stage1_reco_stage2_20240926T110118_keepup.root
\tHists: np04hd_raw_run029081_1087_dataflow1_datawriter_0_20240905T102629_reco_stage1_reco_stage2_20240926T110118_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
