justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263757.134@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID263757.134@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3448
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-26 13:23:33
SiteUK_Durham
EntryDUNE_UK_SGridDurham_ce4
Last heartbeat2024-09-26 14:52:35
From worker nodeHostnamen258.dur.scotgrid.ac.uk
cpuinfoAMD EPYC 7763 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit171000 (47 hours)
Inner Apptainer?True
Job statejobscript_error
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-26 13:24:57
Input fileshd-protodune:np04hd_raw_run029092_0922_dataflow4_datawriter_0_20240906T074556.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Outputting started 
Output files
Finished2024-09-26 14:52:35
Saved logsjustin-logs:263757.134-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

efficient p-value for these numbers
----view 0: 0.000420928 0.000172973 0.000252962 0.000380278 0.000380039 0.000380278 0.000380278 0.000380278 0.000380278 0.000380039 0.000380278 
----view 1: 2.37226e-05 2.37226e-05 2.37226e-05 2.3663e-05 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 0.000265062 9.95994e-05 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000172973 0.000252962 0.000380278 0.000380039 0.000380278 0.000380278 0.000380278 0.000380278 0.000380039 0.000380278 0.000380278 
----view 1: 2.37226e-05 2.37226e-05 2.3663e-05 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 0.000265062 9.95994e-05 1.29342e-05 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000252962 0.000380278 0.000380039 0.000380278 0.000380278 0.000380278 0.000380278 0.000380039 0.000380278 0.000380278 0.00106597 
----view 1: 2.37226e-05 2.3663e-05 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 0.000265062 9.95994e-05 1.29342e-05 1.28746e-05 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000380278 0.000380039 0.000380278 0.000380278 0.000380278 0.000380278 0.000380039 0.000380278 0.000380278 0.00106597 0 
----view 1: 2.3663e-05 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 0.000265062 9.95994e-05 1.29342e-05 1.28746e-05 1.28746e-05 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000380039 0.000380278 0.000380278 0.000380278 0.000380278 0.000380039 0.000380278 0.000380278 0.00106597 0 0 
----view 1: 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 2.37226e-05 0.000265062 9.95994e-05 1.29342e-05 1.28746e-05 1.28746e-05 1.29342e-05 
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

no of hits closeby  1   event 221508 TrkackID 2 startx, y, z -43.2981 434.902 14.2205  wireno, peakt tpcno 116 6.69068 6 dist 0min T, max_T1.94523 1789.61
no of hits closeby  1   event 221508 TrkackID 2 startx, y, z -43.2981 434.902 14.2205  wireno, peakt tpcno 116 6.69068 6 dist 0min T, max_T1.94523 1789.61
Begin processing the 30th record. run: 29092 subRun: 1 event: 221516 at 26-Sep-2024 15:50:54 BST
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 67533 traces, 67533 summary
Retagger: tagging trace set: gauss with 48771 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 48771 traces tagged "gauss"
FrameSaver: q=1.0292e+06 n=852035 tag=gauss
wclsFrameSaver: saving 67533 traces tagged "wiener"
FrameSaver: q=1.10137e+06 n=759388 tag=wiener
2294 hits in TPC 0
2425 hits in TPC 1
5190 hits in TPC 2
3740 hits in TPC 3
4926 hits in TPC 4
6920 hits in TPC 5
4543 hits in TPC 6
2644 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 42 XUs and 26 XVs -> 2 XUVs
C:0 T:2 12318 XUs and 33342 XVs -> 737 XUVs
C:0 T:3 97 XUs and 157 XVs -> 5 XUVs
C:0 T:4 158 XUs and 184 XVs -> 3 XUVs
C:0 T:5 22184 XUs and 19758 XVs -> 618 XUVs
C:0 T:6 20928 XUs and 26220 XVs -> 1021 XUVs
C:0 T:7 70 XUs and 111 XVs -> 5 XUVs
2391 XUVs total
1406 collection wire objects
2391 potential space points
Neighbour search...
52967 tests to find 31180 neighbours
Iterating with no regularization...
Begin: 7.92445e+06
0 7.48365e+06
1 7.45227e+06
2 7.44982e+06
Now with regularization...
Begin: 6.95403e+06
0 6.95298e+06
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

no of hits closeby  1   event 221516 TrkackID 5 startx, y, z -47.4149 587.481 302.118  wireno, peakt tpcno 448 4684.43 6 dist 3.86382min T, max_T4359.86 4976.85
no of hits closeby  1   event 221516 TrkackID 5 startx, y, z -47.4149 587.481 302.118  wireno, peakt tpcno 448 4684.43 6 dist 3.86382min T, max_T4359.86 4976.85
26-Sep-2024 15:51:59 BST  Closed output file "np04hd_raw_run029092_0922_dataflow4_datawriter_0_20240906T074556_reco_stage1_reco_stage2_20240926T145159_keepup.root"
26-Sep-2024 15:51:59 BST  Closed input file "np04hd_raw_run029092_0922_dataflow4_datawriter_0_20240906T074556_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              48.6719       75.8129       121.711       72.7489       15.6865        30     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000395045   0.00167301     0.0026806    0.00196068    0.000694414      30     
produce:fembfilter:PDHDFEMBFilter                       0.5183       0.571977      0.811662      0.533794      0.0914671       30     
produce:wclsdatahd:WireCellToolkit                      28.4505       31.9364       43.7811       29.6954       4.92882        30     
produce:gaushit:GausHitFinder                          0.359134      0.585895       1.07757      0.577972      0.157897        30     
produce:nhitsfilter:NumberOfHitsFilter                0.000207621   0.000339735   0.000541801   0.000329551   7.72416e-05      30     
produce:reco3d:SpacePointSolver                        0.163821      0.850372       3.2695       0.800777      0.608517        30     
produce:hitpdune:DisambigFromSpacePoints               0.160026      0.433055       1.17126       0.39692      0.208149        30     
produce:pandora:StandardPandora                         2.75233       10.691        28.4816       9.49549       5.43638        30     
produce:pandoraWriter:StandardPandora                  0.0596062     0.126569      0.217623      0.126689      0.036984        30     
produce:pandoraTrack:LArPandoraTrackCreation           0.379253      0.938633       1.53835      0.894923      0.336813        30     
produce:pandoraShower:LArPandoraShowerCreation         0.301689       1.13568       2.29282       1.09961      0.469286        30     
produce:pandoracalo:Calorimetry                        0.204616      0.522375      0.893995      0.514633      0.188433        30     
produce:pandoracalonosce:Calorimetry                   0.177393      0.462321      0.785772      0.463998      0.160609        30     
produce:pandorapid:Chi2ParticleID                     0.00110926    0.00351014     0.0562434    0.00155979    0.00980775       30     
produce:pandoraShowercalo:ShowerCalorimetry             0.24431       1.08978       2.33711       1.03653      0.467097        30     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.241593       1.05906       2.27802       1.0086       0.454898        30     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0639441     0.118751       0.28714      0.106912      0.0466624       30     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0235629     0.0408338     0.084598      0.0371073     0.013717        30     
produce:emtrkmichelid:EmTrackMichelId                   13.6524       23.8596       42.138        23.9879       5.76799        30     
[art]:TriggerResults:TriggerResultInserter            1.7463e-05    2.45419e-05   0.000115318   2.0404e-05    1.74113e-05      30     
end_path:out1:RootOutput                               4.459e-06    9.72167e-06   3.6869e-05     8.676e-06    5.37757e-06      30     
end_path:michelremoving:michelremoving                 0.0167181     0.0924518     0.461361      0.0266325     0.104751        30     
end_path:michelremovinggnocchi:michelremoving          0.0139469     0.0904039     0.468577      0.0276971     0.104666        30     
end_path:calibanagnocchi:CalibAnaTree                  0.178399       0.38466      0.642473      0.371706      0.125425        30     
end_path:out1:RootOutput(write)                        0.455063      0.715048       1.07839      0.675231      0.170262        30     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4928.79 MB
  Peak resident set size usage (VmHWM): 2553.25 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029092_0922_dataflow4_datawriter_0_20240906T074556_reco_stage1_reco_stage2_20240926T145159_keepup.root
\tHists: np04hd_raw_run029092_0922_dataflow4_datawriter_0_20240906T074556_reco_stage1_reco_stage2_20240926T145159_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
justIN time: 2024-09-29 09:28:44 UTC       justIN version: 01.01.08