justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 78369.0@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID78369.0@justin-prod-sched02.dune.hep.ac.uk
Workflow ID3461
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-26 09:04:07
SiteUK_Sheffield
EntryDUNE_UK_Sheffield_lcgce1
Last heartbeat2024-09-26 11:02:41
From worker nodeHostnamewn035.hep
cpuinfoAMD EPYC 7352 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job statejobscript_error
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-26 09:46:35
Input fileshd-protodune:np04hd_raw_run027298_0138_dataflow2_datawriter_0_20240619T144514.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Outputting started 
Output files
Finished2024-09-26 11:02:41
Saved logsjustin-logs:78369.0-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

938 0.000432923 
----view 1: 0.0184469 0.0145968 0.044434 0.0194923 0.0186132 0.0200196 0.0110815 0.00999065 0.0184286 0.0236279 0.0180647 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432938 
----view 1: 0.0145968 0.044434 0.0194923 0.0186132 0.0200196 0.0110815 0.00999065 0.0184286 0.0236279 0.0180647 0.0202456 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000432938 0.000432923 0.000432938 0.000432923 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 
----view 1: 0.044434 0.0194923 0.0186132 0.0200196 0.0110815 0.00999065 0.0184286 0.0236279 0.0180647 0.0202456 0.0201653 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000432923 0.000432938 0.000432923 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432923 
----view 1: 0.0194923 0.0186132 0.0200196 0.0110815 0.00999065 0.0184286 0.0236279 0.0180647 0.0202456 0.0201653 0.0226332 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000432938 0.000432923 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432923 0.000432938 
----view 1: 0.0186132 0.0200196 0.0110815 0.00999065 0.0184286 0.0236279 0.0180647 0.0202456 0.0201653 0.0226332 0.0258587 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000432923 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432923 0.000432938 0.000432923 
----view 1: 0.0200196 0.0110815 0.00999065 0.0184286 0.0236279 0.0180647 0.0202456 0.0201653 0.0226332 0.0258587 0.0232578 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432938 0.000432923 0.000432923 0.000432938 0.000432923 0.000432938 
----view 1: 0.0110815 0.00999065 0.0184286 0.0236279 0.0180647 0.0202456 0.0201653 0.0226332 0.0258587 0.0232578 0.0207189 
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

Begin processing the 35th record. run: 27298 subRun: 1 event: 19458 at 26-Sep-2024 11:59:58 BST
RawFrameSource: got 10240 raw::RawDigit objects
	input nticks=5859 keeping as is
Retagger: tagging trace set: wiener with 76364 traces, 76364 summary
Retagger: tagging trace set: gauss with 56522 traces, 0 summary
wclsFrameSaver saving cooked to 5859 ticks
wclsFrameSaver: saving 56522 traces tagged "gauss"
FrameSaver: q=2.47084e+06 n=1026100 tag=gauss
wclsFrameSaver: saving 76364 traces tagged "wiener"
FrameSaver: q=2.66136e+06 n=935089 tag=wiener
2812 hits in TPC 0
4762 hits in TPC 1
8690 hits in TPC 2
4256 hits in TPC 3
5410 hits in TPC 4
9475 hits in TPC 5
4957 hits in TPC 6
2779 hits in TPC 7
0 X, 0 U, 0 V bad channels
Finding XUV coincidences...
C:0 T:1 7100 XUs and 9849 XVs -> 461 XUVs
C:0 T:2 30966 XUs and 22795 XVs -> 1295 XUVs
C:0 T:3 207 XUs and 292 XVs -> 2 XUVs
C:0 T:4 148 XUs and 232 XVs -> 12 XUVs
C:0 T:5 29468 XUs and 28480 XVs -> 1119 XUVs
C:0 T:6 11063 XUs and 8761 XVs -> 493 XUVs
C:0 T:7 292 XUs and 344 XVs -> 9 XUVs
3391 XUVs total
1949 collection wire objects
3391 potential space points
Neighbour search...
49725 tests to find 27388 neighbours
Iterating with no regularization...
Begin: 8.51184e+07
0 7.7889e+07
1 7.74671e+07
2 7.7451e+07
Now with regularization...
Begin: 7.45454e+07
0 7.45416e+07
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

26-Sep-2024 12:01:33 BST  Closed output file "np04hd_raw_run027298_0138_dataflow2_datawriter_0_20240619T144514_reco_stage1_reco_stage2_20240926T110133_keepup.root"
26-Sep-2024 12:01:33 BST  Closed input file "np04hd_raw_run027298_0138_dataflow2_datawriter_0_20240619T144514_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              55.6102       88.2199       136.453       87.4155       18.3054        35     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000305853   0.00159811    0.00343687    0.00142773    0.000905757      35     
produce:fembfilter:PDHDFEMBFilter                      0.563967      0.598625      0.719242      0.584155      0.0373141       35     
produce:wclsdatahd:WireCellToolkit                      34.0058       37.7478       39.9235       37.9864       1.61001        35     
produce:gaushit:GausHitFinder                           0.32641      0.610939      0.919162      0.591336      0.163585        35     
produce:nhitsfilter:NumberOfHitsFilter                0.000170685   0.000369003   0.000612578   0.000357903   0.000102046      35     
produce:reco3d:SpacePointSolver                        0.0977125      0.7271        2.05158      0.602487      0.457931        35     
produce:hitpdune:DisambigFromSpacePoints               0.105917      0.452364       1.02394      0.480005      0.210518        35     
produce:pandora:StandardPandora                         3.50335       13.6953       47.7917       10.576        8.21389        35     
produce:pandoraWriter:StandardPandora                  0.0629952     0.144038      0.226023      0.143625      0.0413026       35     
produce:pandoraTrack:LArPandoraTrackCreation           0.444619       1.27486       4.08289       1.2201       0.688668        35     
produce:pandoraShower:LArPandoraShowerCreation         0.320684       1.57146       5.99874       1.47568       1.04519        35     
produce:pandoracalo:Calorimetry                        0.215936      0.637695       1.96143      0.619508      0.316279        35     
produce:pandoracalonosce:Calorimetry                   0.195546      0.565075       1.54928      0.563492      0.265071        35     
produce:pandorapid:Chi2ParticleID                     0.00103413    0.00175134    0.00308741    0.00164613    0.000495384      35     
produce:pandoraShowercalo:ShowerCalorimetry            0.292953       1.38567       4.50288       1.41397      0.804912        35     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.284849       1.31285       4.36905       1.21963      0.802417        35     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0541701     0.118264      0.197289      0.114935      0.0364103       35     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.018863      0.041833      0.0707391     0.0413238     0.0118086       35     
produce:emtrkmichelid:EmTrackMichelId                   12.8072       25.9047       40.7442       25.9195       6.3559         35     
[art]:TriggerResults:TriggerResultInserter            1.3305e-05    2.43888e-05   9.5492e-05    2.0288e-05    1.47435e-05      35     
end_path:out1:RootOutput                               3.316e-06    4.86849e-06    2.622e-05     4.138e-06    3.69844e-06      35     
end_path:michelremoving:michelremoving                 0.0132842     0.0668694     0.322614      0.0284072     0.0720555       35     
end_path:michelremovinggnocchi:michelremoving         0.00980199     0.0631317     0.312156      0.0244972     0.0688474       35     
end_path:calibanagnocchi:CalibAnaTree                  0.143836      0.461245       1.33841      0.450171      0.234829        35     
end_path:out1:RootOutput(write)                        0.335204      0.708388       1.16708      0.688857      0.194916        35     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5007.84 MB
  Peak resident set size usage (VmHWM): 2420.47 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run027298_0138_dataflow2_datawriter_0_20240619T144514_reco_stage1_reco_stage2_20240926T110133_keepup.root
\tHists: np04hd_raw_run027298_0138_dataflow2_datawriter_0_20240619T144514_reco_stage1_reco_stage2_20240926T110133_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
justIN time: 2024-11-17 03:15:35 UTC       justIN version: 01.01.09