Jobsub ID 263276.90@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263276.90@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:54:33 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce02 |
| Last heartbeat | 2024-09-26 11:22:18 |
| From worker node | Hostname | ccwcondor0635 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:55:55 |
| Input files | hd-protodune:np04hd_raw_run029081_0140_dataflow4_datawriter_0_20240904T193815.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:22:18 |
| Saved logs | justin-logs:263276.90-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
0 T:7 587 XUs and 264 XVs -> 21 XUVs
5112 XUVs total
1395 collection wire objects
5112 potential space points
Neighbour search...
363572 tests to find 191950 neighbours
Iterating with no regularization...
Begin: 1.02536e+07
0 9.2901e+06
1 9.2044e+06
2 9.192e+06
3 9.18834e+06
Now with regularization...
Begin: 8.42868e+06
0 8.42623e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 5 event 33820 TrkackID 4 startx, y, z -154.035 477.205 404.845 wireno, peakt tpcno 52 1752.01 6 dist 1.24043min T, max_T4.98606 1975.9
no of hits closeby 5 event 33820 TrkackID 4 startx, y, z -154.035 477.205 404.845 wireno, peakt tpcno 52 1752.01 6 dist 1.24043min T, max_T4.98606 1975.9
Begin processing the 29th record. run: 29081 subRun: 1 event: 33828 at 26-Sep-2024 13:03:54 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 47668 traces, 47668 summary
Retagger: tagging trace set: gauss with 37083 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 37083 traces tagged "gauss"
FrameSaver: q=843820 n=728391 tag=gauss
wclsFrameSaver: saving 47668 traces tagged "wiener"
FrameSaver: q=895302 n=669772 tag=wiener
2166 hits in TPC 0
3000 hits in TPC 1
5654 hits in TPC 2
3734 hits in TPC 3
2086 hits in TPC 4
2365 hits in TPC 5
4028 hits in TPC 6
2225 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3044 XUs and 459 XVs -> 73 XUVs
C:0 T:2 20244 XUs and 46822 XVs -> 1940 XUVs
C:0 T:3 110 XUs and 185 XVs -> 4 XUVs
C:0 T:4 136 XUs and 132 XVs -> 1 XUVs
C:0 T:5 2881 XUs and 2377 XVs -> 79 XUVs
C:0 T:6 15820 XUs and 33917 XVs -> 830 XUVs
C:0 T:7 164 XUs and 210 XVs -> 10 XUVs
2937 XUVs total
1105 collection wire objects
2937 potential space points
Neighbour search...
203893 tests to find 106732 neighbours
Iterating with no regularization...
Begin: 4.04699e+06
0 3.79411e+06
1 3.77258e+06
2 3.76899e+06
Now with regularization...
Begin: 3.50882e+06
0 3.50721e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 6 event 33828 TrkackID 5 startx, y, z 96.0343 591.701 254.702 wireno, peakt tpcno 53 4452.42 5 dist 3.07063min T, max_T4422.73 5730.28
no of hits closeby 6 event 33828 TrkackID 5 startx, y, z 96.0343 591.701 254.702 wireno, peakt tpcno 53 4452.42 5 dist 3.07063min T, max_T4422.73 5730.28
Begin processing the 30th record. run: 29081 subRun: 1 event: 33836 at 26-Sep-2024 13:04:57 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 76919 traces, 76919 summary
Retagger: tagging trace set: gauss with 55773 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 55773 traces tagged "gauss"
FrameSaver: q=1.20307e+06 n=1004129 tag=gauss
wclsFrameSaver: saving 76919 traces tagged "wiener"
FrameSaver: q=1.29306e+06 n=899678 tag=wiener
3816 hits in TPC 0
3775 hits in TPC 1
5225 hits in TPC 2
3301 hits in TPC 3
4021 hits in TPC 4
8711 hits in TPC 5
7130 hits in TPC 6
3429 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 328 XUs and 762 XVs -> 3 XUVs
C:0 T:2 16076 XUs and 16777 XVs -> 682 XUVs
C:0 T:3 240 XUs and 508 XVs -> 4 XUVs
C:0 T:4 795 XUs and 853 XVs -> 48 XUVs
C:0 T:5 30155 XUs and 25177 XVs -> 659 XUVs
C:0 T:6 39627 XUs and 23861 XVs -> 1055 XUVs
C:0 T:7 435 XUs and 282 XVs -> 8 XUVs
2459 XUVs total
1408 collection wire objects
2459 potential space points
Neighbour search...
30451 tests to find 18476 neighbours
Iterating with no regularization...
Begin: 8.02571e+06
0 7.53845e+06
1 7.51014e+06
2 7.5087e+06
Now with regularization...
Begin: 6.89239e+06
0 6.89195e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:06:48 CEST Closed output file "np04hd_raw_run029081_0140_dataflow4_datawriter_0_20240904T193815_reco_stage1_reco_stage2_20240926T110648_keepup.root"
26-Sep-2024 13:06:48 CEST Closed input file "np04hd_raw_run029081_0140_dataflow4_datawriter_0_20240904T193815_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 62.592 106.57 191.434 100.895 30.3531 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000689457 0.00327999 0.00649111 0.00329099 0.00112324 30
produce:fembfilter:PDHDFEMBFilter 0.525083 0.700296 0.84859 0.739908 0.0930959 30
produce:wclsdatahd:WireCellToolkit 32.7663 42.1088 48.5117 43.4732 4.46618 30
produce:gaushit:GausHitFinder 0.390414 0.755997 1.45426 0.766331 0.232058 30
produce:nhitsfilter:NumberOfHitsFilter 0.000223758 0.000377947 0.000651015 0.000352057 9.50519e-05 30
produce:reco3d:SpacePointSolver 0.116873 1.13241 4.26746 0.730105 0.930986 30
produce:hitpdune:DisambigFromSpacePoints 0.191883 0.622448 1.84713 0.579497 0.335821 30
produce:pandora:StandardPandora 3.68729 14.7871 51.6202 13.0413 9.64339 30
produce:pandoraWriter:StandardPandora 0.0774675 0.168819 0.279702 0.162754 0.0462699 30
produce:pandoraTrack:LArPandoraTrackCreation 0.706667 2.16245 7.13355 1.94384 1.19082 30
produce:pandoraShower:LArPandoraShowerCreation 0.492114 2.8005 7.74977 2.59697 1.67386 30
produce:pandoracalo:Calorimetry 0.369447 1.06219 2.81028 0.920124 0.607024 30
produce:pandoracalonosce:Calorimetry 0.280766 0.981386 3.26606 0.861378 0.652141 30
produce:pandorapid:Chi2ParticleID 0.00124147 0.00231286 0.00434492 0.00221278 0.000762357 30
produce:pandoraShowercalo:ShowerCalorimetry 0.45453 2.2903 7.53379 1.83354 1.66115 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.435222 2.1898 6.77159 1.88685 1.48579 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.05723 0.158653 0.284665 0.164334 0.0590503 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0264151 0.0587653 0.10869 0.0622551 0.0201426 30
produce:emtrkmichelid:EmTrackMichelId 17.3308 32.6594 61.4838 32.3597 10.3279 30
[art]:TriggerResults:TriggerResultInserter 3.2651e-05 7.13985e-05 0.00022533 6.77515e-05 3.27815e-05 30
end_path:out1:RootOutput 1.3074e-05 2.01042e-05 4.4884e-05 1.83845e-05 6.21352e-06 30
end_path:michelremoving:michelremoving 0.0189277 0.11319 0.507043 0.0540422 0.1146 30
end_path:michelremovinggnocchi:michelremoving 0.0188524 0.105536 0.518204 0.0458829 0.114999 30
end_path:calibanagnocchi:CalibAnaTree 0.174543 0.6783 1.99775 0.606186 0.385387 30
end_path:out1:RootOutput(write) 0.408987 0.805635 1.45602 0.825325 0.241682 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4916.15 MB
Peak resident set size usage (VmHWM): 2340.91 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0140_dataflow4_datawriter_0_20240904T193815_reco_stage1_reco_stage2_20240926T110648_keepup.root
\tHists: np04hd_raw_run029081_0140_dataflow4_datawriter_0_20240904T193815_reco_stage1_reco_stage2_20240926T110648_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
