Jobsub ID 78687.147@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 78687.147@justin-prod-sched02.dune.hep.ac.uk |
| Workflow ID | 3447 |
| Stage ID | 1 |
| User name | calcuttj@fnal.gov |
| HTCondor Group | group_dune.prod_mcsim |
| Requested | Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 18000 (5 hours) |
| Submitted time | 2024-09-26 11:27:14 |
| Site | UK_Manchester |
| Entry | UBoone_T2_UK_Manchester_ce02 |
| Last heartbeat | 2024-09-26 14:22:03 |
| From worker node | Hostname | wn2206301.tier2.hep.manchester.ac.uk |
| cpuinfo | Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz |
| OS release | Scientific Linux release 7.9 (Nitrogen) |
| Processors | 1 |
| RSS bytes | 4193255424 (3999 MiB) |
| Wall seconds limit | 257400 (71 hours) |
| Inner Apptainer? | True |
| Job state | jobscript_error |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk |
| Started | 2024-09-26 12:18:04 |
| Input files | hd-protodune:np04hd_raw_run029081_0486_dataflow7_datawriter_0_20240905T003411.hdf5
|
| Jobscript | Exit code | 1 |
| Real time | 0m (0s) |
| CPU time | 0m (0s = 0%) |
| Outputting started | |
| Output files | |
| Finished | 2024-09-26 14:22:03 |
| Saved logs | justin-logs:78687.147-justin-prod-sched02.dune.hep.ac.uk.logs.tgz |
| List job events Wrapper job log |
Jobscript log (last 10,000 characters)
------------------------
------------------------------------------------------------------------------------------------
Begin processing the 29th record. run: 29081 subRun: 1 event: 116895 at 26-Sep-2024 15:13:12 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 82453 traces, 82453 summary
Retagger: tagging trace set: gauss with 60221 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 60221 traces tagged "gauss"
FrameSaver: q=1.39896e+06 n=1103762 tag=gauss
wclsFrameSaver: saving 82453 traces tagged "wiener"
FrameSaver: q=1.48702e+06 n=981013 tag=wiener
3410 hits in TPC 0
3004 hits in TPC 1
8697 hits in TPC 2
6466 hits in TPC 3
2011 hits in TPC 4
4152 hits in TPC 5
7500 hits in TPC 6
6108 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 109 XUs and 210 XVs -> 0 XUVs
C:0 T:2 36541 XUs and 91286 XVs -> 1877 XUVs
C:0 T:3 20 XUs and 48 XVs -> 1 XUVs
C:0 T:4 30 XUs and 8 XVs -> 0 XUVs
C:0 T:5 15147 XUs and 7409 XVs -> 273 XUVs
C:0 T:6 42026 XUs and 58167 XVs -> 3113 XUVs
C:0 T:7 183 XUs and 211 XVs -> 8 XUVs
5272 XUVs total
1853 collection wire objects
5272 potential space points
Neighbour search...
727652 tests to find 333666 neighbours
Iterating with no regularization...
Begin: 1.09043e+07
0 1.02098e+07
1 1.01761e+07
2 1.0172e+07
Now with regularization...
Begin: 9.61852e+06
0 9.61714e+06
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 5 event 116895 TrkackID 30 startx, y, z 22.0051 591.786 196.372 wireno, peakt tpcno 122 4983.48 5 dist 0.347157min T, max_T4920 5188.11
no of hits closeby 5 event 116895 TrkackID 30 startx, y, z 22.0051 591.786 196.372 wireno, peakt tpcno 122 4983.48 5 dist 0.347157min T, max_T4920 5188.11
Begin processing the 30th record. run: 29081 subRun: 1 event: 116903 at 26-Sep-2024 15:17:25 BST
RawFrameSource: got 10240 raw::RawDigit objects
input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 78723 traces, 78723 summary
Retagger: tagging trace set: gauss with 58362 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 58362 traces tagged "gauss"
FrameSaver: q=1.33e+06 n=1140466 tag=gauss
wclsFrameSaver: saving 78723 traces tagged "wiener"
FrameSaver: q=1.42824e+06 n=1041527 tag=wiener
3332 hits in TPC 0
3740 hits in TPC 1
7909 hits in TPC 2
4903 hits in TPC 3
4580 hits in TPC 4
6414 hits in TPC 5
8308 hits in TPC 6
6484 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 3453 XUs and 2031 XVs -> 41 XUVs
C:0 T:2 51109 XUs and 53911 XVs -> 3354 XUVs
C:0 T:3 210 XUs and 335 XVs -> 14 XUVs
C:0 T:4 170 XUs and 315 XVs -> 4 XUVs
C:0 T:5 21887 XUs and 26388 XVs -> 512 XUVs
C:0 T:6 42168 XUs and 49798 XVs -> 1486 XUVs
C:0 T:7 239 XUs and 281 XVs -> 6 XUVs
5417 XUVs total
1842 collection wire objects
5417 potential space points
Neighbour search...
385655 tests to find 214308 neighbours
Iterating with no regularization...
Begin: 2.58756e+07
0 2.4198e+07
1 2.41103e+07
2 2.41027e+07
Now with regularization...
Begin: 2.22817e+07
0 2.2267e+07
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
no of hits closeby 3 event 116903 TrkackID 2 startx, y, z -135.832 459.495 4.27341 wireno, peakt tpcno 473 1365.94 6 dist 0min T, max_T1342.05 3217.03
no of hits closeby 1 event 116903 TrkackID 3 startx, y, z 345.392 222.388 444 wireno, peakt tpcno 1 5306.35 5 dist 2.99757min T, max_T1338 5762.5
no of hits closeby 3 event 116903 TrkackID 2 startx, y, z -135.832 459.495 4.27341 wireno, peakt tpcno 473 1365.94 6 dist 0min T, max_T1342.05 3217.03
no of hits closeby 1 event 116903 TrkackID 3 startx, y, z 345.392 222.388 444 wireno, peakt tpcno 1 5306.35 5 dist 2.99757min T, max_T1338 5762.5
26-Sep-2024 15:21:21 BST Closed output file "np04hd_raw_run029081_0486_dataflow7_datawriter_0_20240905T003411_reco_stage1_reco_stage2_20240926T142121_keepup.root"
26-Sep-2024 15:21:21 BST Closed input file "np04hd_raw_run029081_0486_dataflow7_datawriter_0_20240905T003411_reco_stage1.root"
========================================================================================================================================
TimeTracker printout (sec) Min Avg Max Median RMS nEvts
========================================================================================================================================
Full event 119.314 179.552 252.697 181.8 34.6665 30
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read) 0.000717927 0.00324948 0.0073948 0.00294321 0.00159727 30
produce:fembfilter:PDHDFEMBFilter 0.728817 1.07688 1.27746 1.10677 0.124605 30
produce:wclsdatahd:WireCellToolkit 59.5411 70.8685 83.9682 70.4636 5.56952 30
produce:gaushit:GausHitFinder 0.664427 1.25861 1.8304 1.23201 0.292631 30
produce:nhitsfilter:NumberOfHitsFilter 0.000325916 0.000592906 0.00104699 0.000575718 0.000127987 30
produce:reco3d:SpacePointSolver 0.230431 1.72167 7.61098 1.33587 1.4345 30
produce:hitpdune:DisambigFromSpacePoints 0.197446 1.07685 2.48411 1.03797 0.508167 30
produce:pandora:StandardPandora 7.49935 22.8427 38.2839 20.0053 9.72542 30
produce:pandoraWriter:StandardPandora 0.136689 0.262958 0.436897 0.239432 0.0730888 30
produce:pandoraTrack:LArPandoraTrackCreation 1.48847 3.7851 7.99165 3.45321 1.48287 30
produce:pandoraShower:LArPandoraShowerCreation 1.33395 4.81717 11.4532 4.02272 2.51322 30
produce:pandoracalo:Calorimetry 0.697093 1.55393 3.23672 1.39905 0.566511 30
produce:pandoracalonosce:Calorimetry 0.548479 1.41338 3.15871 1.37014 0.539334 30
produce:pandorapid:Chi2ParticleID 0.00175457 0.00310151 0.00667176 0.00307845 0.00090391 30
produce:pandoraShowercalo:ShowerCalorimetry 1.07398 3.90941 10.2383 3.07537 2.1564 30
produce:pandoraShowercalonosce:ShowerCalorimetry 1.01385 3.88953 10.1395 3.3612 2.11062 30
produce:pandoraGnocchiCalo:GnocchiCalorimetry 0.148356 0.258367 0.385009 0.25 0.0621011 30
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry 0.0609939 0.0987357 0.15226 0.101506 0.0225515 30
produce:emtrkmichelid:EmTrackMichelId 33.9274 57.3142 81.7851 57.1187 12.9644 30
[art]:TriggerResults:TriggerResultInserter 5.6867e-05 7.45874e-05 0.000162584 7.007e-05 1.81725e-05 30
end_path:out1:RootOutput 1.5733e-05 2.26687e-05 5.0874e-05 2.09085e-05 6.56992e-06 30
end_path:michelremoving:michelremoving 0.0465926 0.226172 0.81055 0.114361 0.20654 30
end_path:michelremovinggnocchi:michelremoving 0.042678 0.221644 0.795098 0.105834 0.205091 30
end_path:calibanagnocchi:CalibAnaTree 0.567449 1.33961 2.80296 1.302 0.553164 30
end_path:out1:RootOutput(write) 0.85187 1.39047 2.16075 1.3827 0.348548 30
========================================================================================================================================
====================================================================================================
MemoryTracker summary (base-10 MB units used)
Peak virtual memory usage (VmPeak) : 4944.79 MB
Peak resident set size usage (VmHWM): 2609.21 MB
Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0486_dataflow7_datawriter_0_20240905T003411_reco_stage1_reco_stage2_20240926T142121_keepup.root
\tHists: np04hd_raw_run029081_0486_dataflow7_datawriter_0_20240905T003411_reco_stage1_reco_stage2_20240926T142121_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
