justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

Jobsub ID 78692.188@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID78692.188@justin-prod-sched02.dune.hep.ac.uk
Workflow ID3447
Stage ID1
User namecalcuttj@fnal.gov
HTCondor Groupgroup_dune.prod_mcsim
RequestedProcessors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit18000 (5 hours)
Submitted time2024-09-26 11:28:53
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce02
Last heartbeat2024-09-26 14:19:10
From worker nodeHostnamewn1903240.tier2.hep.manchester.ac.uk
cpuinfoIntel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4193255424 (3999 MiB)
Wall seconds limit257400 (71 hours)
Inner Apptainer?True
Job statejobscript_error
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2024-09-26 12:19:42
Input fileshd-protodune:np04hd_raw_run029081_0679_dataflow2_datawriter_0_20240905T035038.hdf5
JobscriptExit code1
Real time0m (0s)
CPU time0m (0s = 0%)
Outputting started 
Output files
Finished2024-09-26 14:19:10
Saved logsjustin-logs:78692.188-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

oincidences...
C:0 T:1 316 XUs and 473 XVs -> 13 XUVs
C:0 T:2 51488 XUs and 96290 XVs -> 3329 XUVs
C:0 T:3 718 XUs and 1584 XVs -> 29 XUVs
C:0 T:4 956 XUs and 511 XVs -> 23 XUVs
C:0 T:5 177411 XUs and 73493 XVs -> 3904 XUVs
C:0 T:6 35212 XUs and 120067 XVs -> 4092 XUVs
C:0 T:7 1317 XUs and 3254 XVs -> 47 XUVs
11437 XUVs total
4040 collection wire objects
11437 potential space points
Neighbour search...
1112433 tests to find 399466 neighbours
Iterating with no regularization...
Begin: 7.62533e+07
0 5.34669e+07
1 5.15616e+07
2 5.14045e+07
3 5.13629e+07
Now with regularization...
Begin: 4.82253e+07
0 4.8192e+07
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0407525 0.0264882 0.028164 0.0332652 0.0166557 0.027148 0.031677 0.0273116 0.0271324 0.0247871 0.0246232 0.0329778 0.0249218 
----view 1: 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0264882 0.028164 0.0332652 0.0166557 0.027148 0.031677 0.0273116 0.0271324 0.0247871 0.0246232 0.0329778 0.0249218 0.0446579 
----view 1: 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.028164 0.0332652 0.0166557 0.027148 0.031677 0.0273116 0.0271324 0.0247871 0.0246232 0.0329778 0.0249218 0.0446579 0.0271627 
----view 1: 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0332652 0.0166557 0.027148 0.031677 0.0273116 0.0271324 0.0247871 0.0246232 0.0329778 0.0249218 0.0446579 0.0271627 0.0253692 
----view 1: 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.020261 0.0129631 0.0636351 0.0793445 0.0232953 0.0201418 0.0275367 0.0235113 0.0227906 0.0213181 0.0203472 0.0214776 0.0241219 
----view 1: 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0129631 0.0636351 0.0793445 0.0232953 0.0201418 0.0275367 0.0235113 0.0227906 0.0213181 0.0203472 0.0214776 0.0241219 0.0433288 
----view 1: 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0636351 0.0793445 0.0232953 0.0201418 0.0275367 0.0235113 0.0227906 0.0213181 0.0203472 0.0214776 0.0241219 0.0433288 0.0426399 
----view 1: 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 
TwoViewTransverseTracksAlgorithm: failed to calculate correlation coefficient p-value for these numbers
----view 0: 0.0793445 0.0232953 0.0201418 0.0275367 0.0235113 0.0227906 0.0213181 0.0203472 0.0214776 0.0241219 0.0433288 0.0426399 0.0576638 
----view 1: 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154722 0.00154728 0.00154722 0.00154722 0.00154722 
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
BdtBeamParticleIdTool::SliceFeatures::GetLeadingCaloHits - empty calo hit list
---INTERPRETED-MATCHING-OUTPUT------------------------------------------------------------------
---SUMMARY--------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

no of hits closeby  0   event 163194 TrkackID 9 startx, y, z 202.808 280.142 368.625  wireno, peakt tpcno 169 9.19008 5 dist 0min T, max_T5 2596.5
no of hits closeby  0   event 163194 TrkackID 9 startx, y, z 202.808 280.142 368.625  wireno, peakt tpcno 169 9.19008 5 dist 0min T, max_T5 2596.5
26-Sep-2024 15:18:35 BST  Closed output file "np04hd_raw_run029081_0679_dataflow2_datawriter_0_20240905T035038_reco_stage1_reco_stage2_20240926T141835_keepup.root"
26-Sep-2024 15:18:35 BST  Closed input file "np04hd_raw_run029081_0679_dataflow2_datawriter_0_20240905T035038_reco_stage1.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              85.7449       170.083       367.891       163.92        47.8359        30     
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.000777389   0.00246095    0.00442388    0.00248783    0.00106168       30     
produce:fembfilter:PDHDFEMBFilter                      0.695521       1.1101        1.35557       1.05973      0.145246        30     
produce:wclsdatahd:WireCellToolkit                      49.4798       68.3509       78.419        69.8881       6.42442        30     
produce:gaushit:GausHitFinder                          0.498981       1.24712       2.45489       1.26765      0.363649        30     
produce:nhitsfilter:NumberOfHitsFilter                0.000299117   0.000614866   0.00117297    0.000587924   0.000177736      30     
produce:reco3d:SpacePointSolver                        0.357935       1.78168       6.35033       1.1305        1.51523        30     
produce:hitpdune:DisambigFromSpacePoints               0.198227       1.01226       3.6089       0.912807      0.571924        30     
produce:pandora:StandardPandora                         7.41787       26.1655       113.121       23.3075       18.4253        30     
produce:pandoraWriter:StandardPandora                  0.0971541     0.269981      0.443194       0.26875      0.0797978       30     
produce:pandoraTrack:LArPandoraTrackCreation           0.739984       3.92541       10.5365       3.84502       1.8977         30     
produce:pandoraShower:LArPandoraShowerCreation         0.660816       5.03743       16.5142       4.4498        3.32026        30     
produce:pandoracalo:Calorimetry                        0.316339       1.61617       4.50856       1.5776       0.775806        30     
produce:pandoracalonosce:Calorimetry                   0.332101       1.51926       4.42804       1.45299      0.759912        30     
produce:pandorapid:Chi2ParticleID                     0.00142647    0.00310922     0.0052121    0.00312959    0.000855127      30     
produce:pandoraShowercalo:ShowerCalorimetry            0.445868       4.11574       15.126        3.0482        2.95602        30     
produce:pandoraShowercalonosce:ShowerCalorimetry       0.442017       4.0977        14.2317       3.72087       2.76016        30     
produce:pandoraGnocchiCalo:GnocchiCalorimetry          0.0802814     0.250043      0.506077      0.245895      0.0928266       30     
produce:pandoraGnocchiCalonosce:GnocchiCalorimetry     0.0304107     0.0977092     0.202068      0.0954228     0.0370052       30     
produce:emtrkmichelid:EmTrackMichelId                   22.6767       46.1568       90.7983       44.9215       12.2207        30     
[art]:TriggerResults:TriggerResultInserter            5.8118e-05     7.331e-05    0.000124266   7.1998e-05    1.19369e-05      30     
end_path:out1:RootOutput                               1.557e-05    2.30487e-05   3.8666e-05    2.2105e-05    5.51329e-06      30     
end_path:michelremoving:michelremoving                 0.0339819     0.201302      0.677811      0.113857      0.169278        30     
end_path:michelremovinggnocchi:michelremoving          0.0311299     0.193597      0.659826       0.10229       0.16837        30     
end_path:calibanagnocchi:CalibAnaTree                  0.351795       1.34688       4.16451       1.1696       0.674086        30     
end_path:out1:RootOutput(write)                        0.733096       1.36941       2.67467       1.31058      0.341336        30     
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 4964.47 MB
  Peak resident set size usage (VmHWM): 2627.19 MB
  Details saved in: 'mem.db'
====================================================================================================
Art has completed and will exit with status 0.
Output files:
\tReco: np04hd_raw_run029081_0679_dataflow2_datawriter_0_20240905T035038_reco_stage1_reco_stage2_20240926T141835_keepup.root
\tHists: np04hd_raw_run029081_0679_dataflow2_datawriter_0_20240905T035038_reco_stage1_reco_stage2_20240926T141835_keepup_hists.root
Forming reco metadata
/cvmfs/larsoft.opensciencegrid.org/products/python/v3_9_15/Linux64bit+3.10-2.17/bin/python: No module named meta_maker
Error in reco metadata
justIN time: 2024-11-17 07:38:02 UTC       justIN version: 01.01.09