justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 263327.61@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID263327.61@justin-prod-sched01.dune.hep.ac.uk
Workflow ID3447
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-26 10:08:48
SiteIT_CNAF
EntryCMSHTPC_T1_IT_CNAF_condor_ce02
Last heartbeat2024-09-26 12:10:51
From worker nodeHostnamecn-leo-161.cr.cnaf.infn.it
cpuinfoIntel(R) Xeon(R) Platinum 8480+
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit216000 (60 hours)
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-26 10:41:13
Input fileshd-protodune:np04hd_raw_run029081_1032_dataflow3_datawriter_0_20240905T092224.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Outputting started 
Output files
Finished2024-09-26 12:10:51
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

0.0130491 0.00954938 0.0133477 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.3517e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 
----view 1: 0.00856519 0.00995994 0.00441813 0.00800848 0.00401413 0.0253972 0.0156159 0.0125508 0.0117002 0.0130491 0.00954938 0.0133477 0.0116174 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.3517e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.3517e-05 
----view 1: 0.00995994 0.00441813 0.00800848 0.00401413 0.0253972 0.0156159 0.0125508 0.0117002 0.0130491 0.00954938 0.0133477 0.0116174 0.00708437 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 4.35151e-05 4.35151e-05 4.35151e-05 4.3517e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.3517e-05 4.35151e-05 
----view 1: 0.00441813 0.00800848 0.00401413 0.0253972 0.0156159 0.0125508 0.0117002 0.0130491 0.00954938 0.0133477 0.0116174 0.00708437 0.00493944 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 4.35151e-05 4.35151e-05 4.3517e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.3517e-05 4.35151e-05 4.3517e-05 
----view 1: 0.00800848 0.00401413 0.0253972 0.0156159 0.0125508 0.0117002 0.0130491 0.00954938 0.0133477 0.0116174 0.00708437 0.00493944 0.0106821 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 4.35151e-05 4.3517e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.3517e-05 4.35151e-05 4.3517e-05 4.35151e-05 
----view 1: 0.00401413 0.0253972 0.0156159 0.0125508 0.0117002 0.0130491 0.00954938 0.0133477 0.0116174 0.00708437 0.00493944 0.0106821 0.0140851 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 4.3517e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.35151e-05 4.3517e-05 4.35151e-05 4.3517e-05 4.35151e-05 4.35151e-05 
----view 1: 0.0253972 0.0156159 0.0125508 0.0117002 0.0130491 0.00954938 0.0133477 0.0116174 0.00708437 0.00493944 0.0106821 0.0140851 0.0132248 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00252044 0.00230026 0.00224793 0.00219381 0.00229204 0.0023371 0.00222111 0.00220907 0.00218797 0.00224507 0.00243187 
----view 1: 0.0282764 0.0152373 0.0143651 0.00869095 0.00963545 0.00834298 0.012898 0.00949216 0.00774837 0.00399733 0.010602 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00230026 0.00224793 0.00219381 0.00229204 0.0023371 0.00222111 0.00220907 0.00218797 0.00224507 0.00243187 0.00240541 
----view 1: 0.0152373 0.0143651 0.00869095 0.00963545 0.00834298 0.012898 0.00949216 0.00774837 0.00399733 0.010602 0.00893712 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.00224793 0.00219381 0.00229204 0.0023371 0.00222111 0.00220907 0.00218797 0.00224507 0.00243187 0.00240541 0.00237989 
----view 1: 0.0143651 0.00869095 0.00963545 0.00834298 0.012898 0.00949216 0.00774837 0.00399733 0.010602 0.00893712 0.0105367 
BdtBeamParticleIdTool::SliceFeatures::GetLArTPCIntercepts - inconsistent number of intercepts between a line and the LArTPC
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

no of hits closeby  5   event 247923 TrkackID 2 startx, y, z 80.351 515.766 3.84213  wireno, peakt tpcno 252 20.0266 5 dist 0min T, max_T6.26741 1344.67
no of hits closeby  0   event 247923 TrkackID 5 startx, y, z 181.549 593.238 414.385  wireno, peakt tpcno 226 3.5 5 dist 0min T, max_T3.5 2305.68
no of hits closeby  5   event 247923 TrkackID 2 startx, y, z 80.351 515.766 3.84213  wireno, peakt tpcno 252 20.0266 5 dist 0min T, max_T6.26741 1344.67
no of hits closeby  0   event 247923 TrkackID 5 startx, y, z 181.549 593.238 414.385  wireno, peakt tpcno 226 3.5 5 dist 0min T, max_T3.5 2305.68
26-Sep-2024 13:46:21 CEST  Closed output file "np04hd_raw_run029081_1032_dataflow3_datawriter_0_20240905T092224_reco_stage1_reco_stage2_20240926T114621_keepup.root"
26-Sep-2024 13:46:21 CEST  Closed input file "np04hd_raw_run029081_1032_dataflow3_datawriter_0_20240905T092224_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              71.9224       89.7068       108.618       89.5724       10.0063        30     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000490346   0.00215619    0.00520313    0.00184885    0.00144207       30     
produce:fembfilter:PDHDFEMBFilter                      0.667793      0.692534      0.854667      0.683237      0.034458        30     
produce:wclsdatahd:WireCellToolkit                      50.8263       56.2059       61.3948       56.2367       2.67327        30     
produce:gaushit:GausHitFinder                          0.295771      0.508151      0.826959      0.498298      0.133779        30     
produce:nhitsfilter:NumberOfHitsFilter                0.000321411   0.000479293   0.00089283    0.000461357   0.000127238      30     
produce:reco3d:SpacePointSolver                        0.107374      0.649941       2.26472      0.588723      0.501699        30     
produce:hitpdune:DisambigFromSpacePoints               0.120052      0.293907      0.536346      0.273238      0.120014        30     
produce:pandora:StandardPandora                         2.94075       8.52266       14.8384       8.26264       2.98712        30     
produce:pandoraWriter:StandardPandora                  0.0847875     0.133572      0.190032      0.130814      0.0279944       30     
produce:pandoraTrack:LArPandoraTrackCreation           0.745863       1.79151       3.41729       1.70005      0.681517        30     
produce:pandoraShower:LArPandoraShowerCreation         0.604842       2.25724       5.22306       2.18254       1.03651        30     
produce:pandoracalo:Calorimetry                        0.354135      0.757503       1.43691      0.712453      0.267489        30     
produce:pandoracalonosce:Calorimetry                   0.283667      0.708214       1.35717      0.687616      0.262097        30     
produce:pandorapid:Chi2ParticleID                     0.00112093    0.00155016    0.00325444    0.00143621     0.0004175       30     
produce:pandoraShowercalo:ShowerCalorimetry            0.471705       1.77439       4.2098        1.75376      0.846421        30     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.476183       1.72782       4.13077       1.70807       0.82308        30     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0534095     0.0861703     0.127424      0.0874289     0.0215603       30     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0230891     0.0365406     0.0550244     0.0361546     0.0087669       30     
produce:emtrkmichelid:EmTrackMichelId                   7.58242       12.0648       18.6767       11.1293       2.78667        30     
[art]:TriggerResults:TriggerResultInserter            4.6707e-05    6.5221e-05    0.000142818   5.85855e-05   2.04227e-05      30     
end_path:out1:RootOutput                              1.7408e-05    2.2802e-05    5.2414e-05    2.00445e-05   6.96393e-06      30     
end_path:michelremoving:michelremoving                 0.0228145     0.0602805     0.247197      0.0402569     0.0539812       30     
end_path:michelremovinggnocchi:michelremoving          0.0206018     0.0561866     0.220194      0.0347959     0.0511205       30     
end_path:calibanagnocchi:CalibAnaTree                  0.192242      0.523466       1.02583      0.493355       0.20858        30     
end_path:out1:RootOutput(write)                        0.346244      0.558987      0.786106       0.55907      0.125796        30     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 5815.73 MB
  Peak resident set size usage (VmHWM): 2661 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_1032_dataflow3_datawriter_0_20240905T092224_reco_stage1_reco_stage2_20240926T114621_keepup.root
\tHists: np04hd_raw_run029081_1032_dataflow3_datawriter_0_20240905T092224_reco_stage1_reco_stage2_20240926T114621_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
justIN time: 2024-11-17 03:11:24 UTC       justIN version: 01.01.09