Jobsub ID 263269.128@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263269.128@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:51:00 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce03 |
| Last heartbeat | 2024-09-26 11:07:36 |
| From worker node | Hostname | ccwcondor0601 |
| cpuinfo | AMD EPYC 7453 28-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:53:30 |
| Input files | hd-protodune:np04hd_raw_run029092_0463_dataflow4_datawriter_0_20240906T004056.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:07:36 |
| Saved logs | justin-logs:263269.128-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
lsFrameSaver: saving 41499 traces tagged "gauss"
FrameSaver: q=880786 n=779624 tag=gauss
wclsFrameSaver: saving 54358 traces tagged "wiener"
FrameSaver: q=936065 n=707298 tag=wiener
3612 hits in TPC 0
2166 hits in TPC 1
5443 hits in TPC 2
2690 hits in TPC 3
2153 hits in TPC 4
3287 hits in TPC 5
5946 hits in TPC 6
3233 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 21 XUs and 8 XVs -> 0 XUVs
C:0 T:2 17879 XUs and 16066 XVs -> 791 XUVs
C:0 T:3 22 XUs and 37 XVs -> 1 XUVs
C:0 T:4 69 XUs and 88 XVs -> 3 XUVs
C:0 T:5 17145 XUs and 12352 XVs -> 290 XUVs
C:0 T:6 17456 XUs and 22250 XVs -> 461 XUVs
C:0 T:7 17 XUs and 28 XVs -> 1 XUVs
1547 XUVs total
940 collection wire objects
1547 potential space points
Neighbour search...
25357 tests to find 15256 neighbours
Iterating with no regularization...
Begin: 7.2245e+06
0 6.94988e+06
1 6.92534e+06
2 6.92332e+06
Now with regularization...
Begin: 6.5704e+06
0 6.56985e+06
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29092 subRun: 1 event: 111372 at 26-Sep-2024 12:54:49 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 63881 traces, 63881 summary
Retagger: tagging trace set: gauss with 46373 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 46373 traces tagged "gauss"
FrameSaver: q=1.02738e+06 n=850085 tag=gauss
wclsFrameSaver: saving 63881 traces tagged "wiener"
FrameSaver: q=1.11427e+06 n=765306 tag=wiener
5894 hits in TPC 0
4914 hits in TPC 1
3544 hits in TPC 2
2917 hits in TPC 3
3487 hits in TPC 4
5921 hits in TPC 5
3174 hits in TPC 6
2370 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3768 XUs and 5100 XVs -> 957 XUVs
C:0 T:2 9883 XUs and 18975 XVs -> 996 XUVs
C:0 T:3 76 XUs and 129 XVs -> 13 XUVs
C:0 T:4 5 XUs and 9 XVs -> 0 XUVs
C:0 T:5 20570 XUs and 38304 XVs -> 4074 XUVs
C:0 T:6 8076 XUs and 28806 XVs -> 2019 XUVs
C:0 T:7 180 XUs and 168 XVs -> 8 XUVs
8067 XUVs total
1674 collection wire objects
8067 potential space points
Neighbour search...
761677 tests to find 420276 neighbours
Iterating with no regularization...
Begin: 9.53251e+06
0 8.17427e+06
1 8.05577e+06
2 8.03378e+06
3 8.02739e+06
Now with regularization...
Begin: 7.46873e+06
0 7.46376e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Begin processing the 30th record. run: 29092 subRun: 1 event: 111380 at 26-Sep-2024 12:56:11 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 66015 traces, 66015 summary
Retagger: tagging trace set: gauss with 47810 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 47810 traces tagged "gauss"
FrameSaver: q=1.00763e+06 n=848500 tag=gauss
wclsFrameSaver: saving 66015 traces tagged "wiener"
FrameSaver: q=1.08667e+06 n=762270 tag=wiener
3874 hits in TPC 0
3768 hits in TPC 1
6055 hits in TPC 2
3002 hits in TPC 3
3264 hits in TPC 4
6183 hits in TPC 5
5175 hits in TPC 6
2979 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 583 XUs and 509 XVs -> 26 XUVs
C:0 T:2 13070 XUs and 33699 XVs -> 588 XUVs
C:0 T:3 87 XUs and 90 XVs -> 2 XUVs
C:0 T:4 272 XUs and 373 XVs -> 25 XUVs
C:0 T:5 29651 XUs and 42550 XVs -> 1049 XUVs
C:0 T:6 35761 XUs and 24529 XVs -> 450 XUVs
C:0 T:7 129 XUs and 96 XVs -> 2 XUVs
2142 XUVs total
1071 collection wire objects
2142 potential space points
Neighbour search...
53216 tests to find 32166 neighbours
Iterating with no regularization...
Begin: 1.0147e+07
0 9.13587e+06
1 9.1008e+06
2 9.09673e+06
Now with regularization...
Begin: 8.83732e+06
0 8.83582e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 12:57:38 CEST Closed output file "np04hd_raw_run029092_0463_dataflow4_datawriter_0_20240906T004056_reco_stage1_reco_stage2_20240926T105738_keepup.root"
26-Sep-2024 12:57:38 CEST Closed input file "np04hd_raw_run029092_0463_dataflow4_datawriter_0_20240906T004056_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 52.3279 93.2125 145.918 93.3467 15.0327 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000590634 0.0028246 0.005329 0.00293801 0.00117834 30
produce:fembfilter:PDHDFEMBFilter 0.524844 0.668446 0.886381 0.65228 0.0866458 30
produce:wclsdatahd:WireCellToolkit 33.5238 38.9763 46.2647 39.1401 2.86391 30
produce:gaushit:GausHitFinder 0.471838 0.780726 4.07029 0.652846 0.631238 30
produce:nhitsfilter:NumberOfHitsFilter 0.000261679 0.000380248 0.00121692 0.00034269 0.000167825 30
produce:reco3d:SpacePointSolver 0.376605 1.18711 3.68541 0.850906 0.904144 29
produce:hitpdune:DisambigFromSpacePoints 0.237402 0.495778 1.22736 0.453692 0.200038 29
produce:pandora:StandardPandora 7.43554 11.9783 21.4529 11.0066 3.535 29
produce:pandoraWriter:StandardPandora 0.10232 0.159709 0.269613 0.160879 0.0379386 29
produce:pandoraTrack:LArPandoraTrackCreation 0.831404 1.84555 4.75653 1.87728 0.704249 29
produce:pandoraShower:LArPandoraShowerCreation 1.05936 2.25039 7.13788 1.98199 1.2093 29
produce:pandoracalo:Calorimetry 0.485558 0.869273 2.3426 0.880275 0.34587 29
produce:pandoracalonosce:Calorimetry 0.404896 0.807623 2.04369 0.787244 0.309634 29
produce:pandorapid:Chi2ParticleID 0.00136891 0.00220805 0.00389451 0.00226768 0.000538343 29
produce:pandoraShowercalo:ShowerCalorimetry 0.945066 1.89234 5.35456 1.61961 0.911422 29
produce:pandoraShowercalonosce:ShowerCalorimetry 0.891734 1.82241 5.74086 1.56797 0.932894 29
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0813687 0.137894 0.257606 0.132274 0.0376996 29
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0277272 0.0524798 0.0984095 0.0500028 0.0139894 29
produce:emtrkmichelid:EmTrackMichelId 21.1591 29.4048 44.849 28.2705 4.58295 29
[art]:TriggerResults:TriggerResultInserter 2.9625e-05 6.37249e-05 0.000105548 6.8518e-05 1.95714e-05 30
end_path:out1:RootOutput 1.1081e-05 2.17265e-05 4.0164e-05 2.23765e-05 5.8001e-06 30
end_path:michelremoving:michelremoving 6.2848e-05 0.077842 0.234466 0.0433067 0.0628732 30
end_path:michelremovinggnocchi:michelremoving 4.5675e-05 0.0701826 0.22019 0.0460165 0.054512 30
end_path:calibanagnocchi:CalibAnaTree 4.5755e-05 0.564641 1.70949 0.536864 0.281455 30
end_path:out1:RootOutput(write) 0.560483 0.722608 1.13187 0.705357 0.117017 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 5019.9 MB
Peak resident set size usage (VmHWM): 2441.49 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0463_dataflow4_datawriter_0_20240906T004056_reco_stage1_reco_stage2_20240926T105738_keepup.root
\tHists: np04hd_raw_run029092_0463_dataflow4_datawriter_0_20240906T004056_reco_stage1_reco_stage2_20240926T105738_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
