Jobsub ID 263276.76@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 263276.76@justin-prod-sched01.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 09:54:33 |
| Site | FR_CCIN2P3 |
| Entry | DUNE_FR_CCIN2P3_cccondorce02 |
| Last heartbeat | 2024-09-26 11:11:34 |
| From worker node | Hostname | ccwcondor0738.in2p3.fr |
| cpuinfo | AMD EPYC 7302 16-Core Processor |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 106200 (29 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 09:56:16 |
| Input files | hd-protodune:np04hd_raw_run029081_0423_dataflow1_datawriter_0_20240904T234125.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 11:11:34 |
| Saved logs | justin-logs:263276.76-justin-prod-sched01.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
-----------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 1 event 101737 TrkackID 1 startx, y, z 247.114 610.903 136.77 wireno, peakt tpcno 477 917.444 5 dist 3.4887min T, max_T894.607 4965.5
no of hits closeby 1 event 101737 TrkackID 1 startx, y, z 247.114 610.903 136.77 wireno, peakt tpcno 477 917.444 5 dist 3.4887min T, max_T894.607 4965.5
Begin processing the 29th record. run: 29081 subRun: 1 event: 101745 at 26-Sep-2024 12:59:21 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 70337 traces, 70337 summary
Retagger: tagging trace set: gauss with 50566 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 50566 traces tagged "gauss"
FrameSaver: q=1.03961e+06 n=864953 tag=gauss
wclsFrameSaver: saving 70337 traces tagged "wiener"
FrameSaver: q=1.1312e+06 n=766072 tag=wiener
4933 hits in TPC 0
5037 hits in TPC 1
6830 hits in TPC 2
3003 hits in TPC 3
1707 hits in TPC 4
4408 hits in TPC 5
5544 hits in TPC 6
3025 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:0 18 XUs and 18 XVs -> 0 XUVs
C:0 T:1 2589 XUs and 2336 XVs -> 63 XUVs
C:0 T:2 45733 XUs and 44238 XVs -> 1328 XUVs
C:0 T:3 44 XUs and 14 XVs -> 0 XUVs
C:0 T:4 76 XUs and 97 XVs -> 4 XUVs
C:0 T:5 8163 XUs and 5849 XVs -> 293 XUVs
C:0 T:6 23813 XUs and 70756 XVs -> 1004 XUVs
C:0 T:7 667 XUs and 285 XVs -> 3 XUVs
2695 XUVs total
1053 collection wire objects
2695 potential space points
Neighbour search...
117217 tests to find 64260 neighbours
Iterating with no regularization...
Begin: 8.77266e+06
0 7.91455e+06
1 7.87867e+06
2 7.87326e+06
Now with regularization...
Begin: 7.45007e+06
0 7.44721e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 0 event 101745 TrkackID 11 startx, y, z 12.7384 369.672 12.8605 wireno, peakt tpcno 59 3546.87 2 dist 268.771min T, max_T2308 3572.74
no of hits closeby 0 event 101745 TrkackID 11 startx, y, z 12.7384 369.672 12.8605 wireno, peakt tpcno 59 3546.87 2 dist 268.771min T, max_T2308 3572.74
Begin processing the 30th record. run: 29081 subRun: 1 event: 101753 at 26-Sep-2024 13:00:52 CEST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 61305 traces, 61305 summary
Retagger: tagging trace set: gauss with 45292 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 45292 traces tagged "gauss"
FrameSaver: q=942386 n=822478 tag=gauss
wclsFrameSaver: saving 61305 traces tagged "wiener"
FrameSaver: q=1.02336e+06 n=747130 tag=wiener
3064 hits in TPC 0
3666 hits in TPC 1
6368 hits in TPC 2
4137 hits in TPC 3
2873 hits in TPC 4
4767 hits in TPC 5
4777 hits in TPC 6
2315 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 919 XUs and 1385 XVs -> 267 XUVs
C:0 T:2 54173 XUs and 63421 XVs -> 2959 XUVs
C:0 T:3 244 XUs and 389 XVs -> 11 XUVs
C:0 T:4 422 XUs and 468 XVs -> 2 XUVs
C:0 T:5 20079 XUs and 15243 XVs -> 3006 XUVs
C:0 T:6 42766 XUs and 52329 XVs -> 1874 XUVs
C:0 T:7 312 XUs and 209 XVs -> 2 XUVs
8121 XUVs total
1626 collection wire objects
8121 potential space points
Neighbour search...
1002267 tests to find 544574 neighbours
Iterating with no regularization...
Begin: 9.99246e+06
0 9.07033e+06
1 9.00909e+06
2 8.99862e+06
3 8.99543e+06
Now with regularization...
Begin: 8.49677e+06
0 8.49357e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
26-Sep-2024 13:02:16 CEST Closed output file "np04hd_raw_run029081_0423_dataflow1_datawriter_0_20240904T234125_reco_stage1_reco_stage2_20240926T110216_keepup.root"
26-Sep-2024 13:02:16 CEST Closed input file "np04hd_raw_run029081_0423_dataflow1_datawriter_0_20240904T234125_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 69.5234 95.4696 119.23 97.3617 12.8417 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000695399 0.00331228 0.00594784 0.00353158 0.00141306 30
produce:fembfilter:PDHDFEMBFilter 0.595327 0.707882 0.90486 0.686915 0.069781 30
produce:wclsdatahd:WireCellToolkit 40.8169 43.9444 46.3704 44.1914 1.41684 30
produce:gaushit:GausHitFinder 0.321198 0.644898 0.951057 0.600877 0.155163 30
produce:nhitsfilter:NumberOfHitsFilter 0.000245381 0.000442044 0.000695439 0.000433931 0.000104395 30
produce:reco3d:SpacePointSolver 0.125293 0.98309 4.50764 0.861739 0.789283 30
produce:hitpdune:DisambigFromSpacePoints 0.109802 0.427583 0.815138 0.418081 0.169521 30
produce:pandora:StandardPandora 4.91411 11.1182 18.5636 11.153 3.59203 30
produce:pandoraWriter:StandardPandora 0.0596006 0.161131 0.273887 0.154191 0.0520398 30
produce:pandoraTrack:LArPandoraTrackCreation 0.263958 1.19659 2.46052 1.15563 0.466884 30
produce:pandoraShower:LArPandoraShowerCreation 0.394635 1.29171 2.25211 1.30581 0.542949 30
produce:pandoracalo:Calorimetry 0.208679 0.57646 0.961546 0.585792 0.178475 30
produce:pandoracalonosce:Calorimetry 0.200256 0.508929 0.827815 0.505313 0.156348 30
produce:pandorapid:Chi2ParticleID 0.00111094 0.00195926 0.00323457 0.00181318 0.000519303 30
produce:pandoraShowercalo:ShowerCalorimetry 0.353104 1.16498 2.1026 1.11522 0.464741 30
produce:pandoraShowercalonosce:ShowerCalorimetry 0.287063 1.14373 2.18017 1.12701 0.446776 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.0720987 0.149211 0.229104 0.146113 0.0389818 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0318049 0.0523271 0.101279 0.048715 0.0163102 30
produce:emtrkmichelid:EmTrackMichelId 15.9267 29.8804 39.7556 30.4537 6.01976 30
[art]:TriggerResults:TriggerResultInserter 2.2893e-05 5.07713e-05 0.000107312 4.75195e-05 2.33491e-05 30
end_path:out1:RootOutput 8.586e-06 1.35689e-05 4.6097e-05 1.05145e-05 8.33175e-06 30
end_path:michelremoving:michelremoving 0.017948 0.0678804 0.228871 0.0381633 0.0594938 30
end_path:michelremovinggnocchi:michelremoving 0.0160761 0.0638922 0.2904 0.0330626 0.063402 30
end_path:calibanagnocchi:CalibAnaTree 0.167399 0.41621 0.744045 0.412913 0.128453 30
end_path:out1:RootOutput(write) 0.455575 0.779281 1.11288 0.803626 0.149403 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4911.99 MB
Peak resident set size usage (VmHWM): 2324.21 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0423_dataflow1_datawriter_0_20240904T234125_reco_stage1_reco_stage2_20240926T110216_keepup.root
\tHists: np04hd_raw_run029081_0423_dataflow1_datawriter_0_20240904T234125_reco_stage1_reco_stage2_20240926T110216_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
