justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263837.6@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID263837.6@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3448
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-26 13:49:39
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2024-09-26 15:49:51
From worker nodeHostnamecn-609-02-01.cr.cnaf.infn.it
cpuinfoIntel(R) Xeon(R) CPU E5-2697 v4 @ 2.30GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job statejobscript_error
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-26 13:50:59
Input fileshd-protodune:np04hd_raw_run029097_0148_dataflow1_datawriter_0_20240906T133436.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Outputting started 
Output files
Finished2024-09-26 15:49:51
Saved logsjustin-logs:263837.6-justin-prod-sched01.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

ire objects
6775 potential space points
Neighbour search...
433763 tests to find 232862 neighbours
Iterating with no regularization...
Begin: 1.17774e+07
0 1.06231e+07
1 1.05143e+07
2 1.04977e+07
3 1.04917e+07
Now with regularization...
Begin: 9.9722e+06
0 9.96855e+06
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000341117 0.000341058 0.000341058 0.000341117 0.000341058 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 0.000308156 0.00030309 
----view 1: 0.0121219 0.0175906 0.00438428 0.00401413 0.00955296 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 0.00504327 0.0131746 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000341058 0.000341058 0.000341117 0.000341058 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 0.000308156 0.00030309 0.00030309 
----view 1: 0.0175906 0.00438428 0.00401413 0.00955296 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 0.00504327 0.0131746 0.0018155 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000341058 0.000341117 0.000341058 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 0.000308156 0.00030309 0.00030309 0.00030309 
----view 1: 0.00438428 0.00401413 0.00955296 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 0.00504327 0.0131746 0.0018155 0.0107136 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000341117 0.000341058 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 0.000308156 0.00030309 0.00030309 0.00030309 0.00030303 
----view 1: 0.00401413 0.00955296 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 0.00504327 0.0131746 0.0018155 0.0107136 0.00162607 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000341117 0.000341058 0.000341058 0.000341117 0.000341058 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 
----view 1: 0.0121219 0.0175906 0.00438428 0.00401413 0.00955296 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000341058 0.000341058 0.000341117 0.000341058 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 0.000308156 
----view 1: 0.0175906 0.00438428 0.00401413 0.00955296 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 0.00504327 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000341058 0.000341117 0.000341058 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 0.000308156 0.00030309 
----view 1: 0.00438428 0.00401413 0.00955296 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 0.00504327 0.0131746 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000341117 0.000341058 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 0.000308156 0.00030309 0.00030309 
----view 1: 0.00401413 0.00955296 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 0.00504327 0.0131746 0.0018155 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000341058 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 0.000308156 0.00030309 0.00030309 0.00030309 
----view 1: 0.00955296 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 0.00504327 0.0131746 0.0018155 0.0107136 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.000332654 0.000320554 0.000320613 0.000320613 0.000320673 0.000320613 0.000308156 0.00030309 0.00030309 0.00030309 0.00030303 
----view 1: 0.00403798 0.010778 0.00629461 0.0083437 0.00525665 0.0249348 0.00504327 0.0131746 0.0018155 0.0107136 0.00162607 
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

26-Sep-2024 17:49:23 CEST  Closed output file "np04hd_raw_run029097_0148_dataflow1_datawriter_0_20240906T133436_reco_stage1_reco_stage2_20240926T154923_keepup.root"
26-Sep-2024 17:49:23 CEST  Closed input file "np04hd_raw_run029097_0148_dataflow1_datawriter_0_20240906T133436_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              118.213       173.531       218.625       181.899       25.6735        30     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000769425   0.00406825    0.00670958    0.00421156    0.00160821       30     
produce:fembfilter:PDHDFEMBFilter                      0.698987       1.04527       1.45054       1.1046       0.154889        30     
produce:wclsdatahd:WireCellToolkit                      55.5646       75.5425       85.6791       76.5656       6.82231        30     
produce:gaushit:GausHitFinder                           0.67959       1.26906       1.69023       1.40348      0.336058        30     
produce:nhitsfilter:NumberOfHitsFilter                0.000330938   0.000767646   0.00411979    0.000640302   0.000647922      30     
produce:reco3d:SpacePointSolver                        0.430929       1.8407        4.40384       1.74932      0.990669        30     
produce:hitpdune:DisambigFromSpacePoints               0.434654       1.4921        2.82346       1.53043      0.676276        30     
produce:pandora:StandardPandora                         11.7034       28.0278       43.926        29.4451       8.46601        30     
produce:pandoraWriter:StandardPandora                  0.185038      0.286696      0.394243      0.305188      0.0555203       30     
produce:pandoraTrack:LArPandoraTrackCreation            1.43233       3.5189        5.48278       3.92703       1.11854        30     
produce:pandoraShower:LArPandoraShowerCreation          1.42772       4.4001        6.88998       4.58002       1.77102        30     
produce:pandoracalo:Calorimetry                        0.610275       1.74537       2.77172       1.93318       0.54925        30     
produce:pandoracalonosce:Calorimetry                   0.580466       1.5645        2.3291        1.72051      0.490437        30     
produce:pandorapid:Chi2ParticleID                     0.00208953    0.00350626    0.00570181    0.00351012    0.000919055      30     
produce:pandoraShowercalo:ShowerCalorimetry            0.959469       3.76609       6.64478       3.9301        1.46315        30     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.984106       3.74692       6.44486       3.71871       1.46499        30     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.090152      0.303788      0.439531      0.324265      0.0862555       30     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0338826     0.105229      0.146204       0.1132       0.0282495       30     
produce:emtrkmichelid:EmTrackMichelId                   23.587        41.9584       64.8025       43.5436       10.1226        30     
[art]:TriggerResults:TriggerResultInserter            4.3576e-05    6.78202e-05   0.000117949   6.23745e-05   1.80407e-05      30     
end_path:out1:RootOutput                              1.1166e-05    1.82021e-05   3.4701e-05    1.67375e-05   5.9708e-06       30     
end_path:michelremoving:michelremoving                 0.0303564     0.133865      0.611902      0.0846708     0.120176        30     
end_path:michelremovinggnocchi:michelremoving          0.0268434      0.12882      0.624879      0.0826199     0.122307        30     
end_path:calibanagnocchi:CalibAnaTree                  0.314033       1.11061       1.84774       1.13878      0.367186        30     
end_path:out1:RootOutput(write)                        0.627228       1.33813       1.85077       1.43656      0.353884        30     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4912.98 MB
  Peak resident set size usage (VmHWM): 2332.97 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029097_0148_dataflow1_datawriter_0_20240906T133436_reco_stage1_reco_stage2_20240926T154923_keepup.root
\tHists: np04hd_raw_run029097_0148_dataflow1_datawriter_0_20240906T133436_reco_stage1_reco_stage2_20240926T154923_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
justIN time: 2024-09-29 09:35:09 UTC       justIN version: 01.01.08