Jobsub ID 108522.156@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108522.156@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-22 22:48:20
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce515
Last heartbeat	2024-11-23 00:16:32
From worker node	Hostname	b9p05p7648.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	stalled
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-22 23:01:56
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_1062_20231203T092006Z_gen_g4_detsim_hitreco__20240508T052611Z_reco2.root
Outputting started	2024-11-23 00:16:32
Output files
Finished	2024-11-23 00:34:32
List job events Wrapper job log
Jobscript log (last 10,000 characters)

ower@BeginModule  23-Nov-2024 01:16:26 CET run: 6410210 subRun: 1 event: 106300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:16:26 CET run: 6410210 subRun: 1 event: 106300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:16:26 CET run: 6410210 subRun: 1 event: 106300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:16:26 CET run: 6410210 subRun: 1 event: 106300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:16:26 CET run: 6410210 subRun: 1 event: 106300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:16:26 CET run: 6410210 subRun: 1 event: 106300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:16:26 CET run: 6410210 subRun: 1 event: 106300
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 639, 612, 568
minwire 0: 949
minwire 1: 344
minwire 2: 1011
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.155798, 0.68407, 0.160132, 
Output 1: 0.0576863, 0.391495, 0.486996, 0.0638227, 
Output 2: 0.13074, 0.86926, 

758
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3089!
23-Nov-2024 01:16:30 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_1062_20231203T092006Z_gen_g4_detsim_hitreco__20240508T052611Z_reco2_reco_data_2024-11-22T_230203Z.root"
23-Nov-2024 01:16:30 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/d2/88/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_1062_20231203T092006Z_gen_g4_detsim_hitreco__20240508T052611Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.05057       42.6477       1031.91       9.41897       144.616        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0169177     0.0866984      3.30942      0.0405853     0.331679        100    
reco:linecluster:LineCluster                           0.0942354      1.53621       59.8817      0.167893       7.45339        100    
reco:trajcluster:TrajCluster                           0.0474306      3.04206       113.512      0.0585855      16.0355        100    
reco:pandora:StandardPandora                           0.0596189      4.44917       129.034       1.2553        15.8039        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000139881   0.00804903     0.237486     0.00144069     0.027854        100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000154512    0.0108264     0.116581     0.00295671     0.0207368       100    
reco:pandoracalo:Calorimetry                          9.6061e-05     0.0590692      2.58191     0.000937978    0.305359        100    
reco:pandorapid:Chi2ParticleID                        3.4061e-05    0.000188159    0.002277     0.000133172   0.000255781      100    
reco:trkshowersplit:TrackShowerHits                   7.6051e-05     0.0584981      1.98732     0.000844991    0.272201        100    
reco:pmtrack:PMAlgTrackMaker                          0.00104808      4.51145       133.903      0.191151       19.3246        100    
reco:pmtrackcalo:Calorimetry                          6.2991e-05     0.0273641      1.44727     0.00100095     0.152582        100    
reco:pmtrackpid:Chi2ParticleID                         2.617e-05    0.000372206    0.0073442    0.000154567   0.000914561      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000168982     3.77671       108.779      0.218677       15.1935        100    
reco:pmtrajfitcalo:Calorimetry                        5.9461e-05     0.0555643      2.38931     0.00142692     0.281251        100    
reco:pmtrajfitpid:Chi2ParticleID                      2.3481e-05    0.000198051   0.00129681    0.000135242   0.000189887      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000856831     6.19997       209.797      0.164558       27.6258        100    
reco:pmtracktccalo:Calorimetry                         6.024e-05     0.0121361     0.486606     0.000944193    0.0549056       100    
reco:pmtracktcpid:Chi2ParticleID                       2.42e-05     0.000297053   0.00311607    0.000157487   0.000494477      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000216033     2.59987       77.1458      0.288023       10.0511        100    
reco:pmtrajfittccalo:Calorimetry                      6.3071e-05     0.0360635      1.33162     0.00116308     0.170313        100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.403e-05    0.000450853   0.00648096    0.000173112   0.00100165       100    
reco:blurredcluster:BlurredClustering                 0.000134552     1.05153       12.1143      0.513094       1.79907        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000152323     11.7489       232.886       2.55318       32.6676        100    
reco:emshower:EMShower                                0.000237963    0.0334576     0.523052     0.00659999     0.066012        100    
reco:cvnmap:CVNMapper                                  2.27e-05      0.0106655     0.0532655     0.0109958    0.00993025       100    
reco:cvneva:CVNEvaluator                               1.733e-05     0.679525       1.19997      0.917317      0.464655        100    
reco:energyrecnumu:EnergyReco                         0.000318894    0.0441907     0.717284      0.0039617     0.122727        100    
reco:energyrecnue:EnergyReco                          0.000143022   0.00242728     0.0613771    0.000528522   0.00706125       100    
reco:energyrecnc:EnergyReco                            0.0017314    0.00485125     0.0633101    0.00326393    0.00698487       100    
reco:energyrecnumurange:EnergyReco                    0.000182863    0.0396623     0.588627     0.00272312     0.111893        100    
reco:energyrecnumumcs:EnergyReco                      0.00138699     0.0425334      0.69323     0.00373807     0.118449        100    
reco:opdec:Deconvolution                               0.0758446     0.284959       1.34678      0.277736      0.137659        100    
reco:ophitspe:OpHitFinderDeco                          0.242931       1.96988       4.50991       1.87609      0.874345        100    
reco:opflash:OpFlashFinder                            0.000254423   0.00152139    0.00617264    0.00143286    0.000791282      100    
reco:rns:RandomNumberSaver                             1.724e-05    4.26634e-05   0.000585168   3.86055e-05   5.54565e-05      100    
reco:anglereconue:NuAngularReco                       0.000149322   0.00186009     0.0558366    0.00039193     0.0061498       100    
reco:anglereconumu:NuAngularReco                      7.4631e-05    0.00164369     0.0583715    0.000234934   0.00634665       100    
reco:anglereconuepfps:NuAngularReco                   0.000124671   0.00186631     0.0671618    0.000370055   0.00712785       100    
reco:anglereconumupfps:NuAngularReco                  0.000103892   0.00183944     0.0710542    0.000296264   0.00750541       100    
[art]:TriggerResults:TriggerResultInserter              8.6e-06     1.37243e-05    5.497e-05    1.2275e-05    6.16789e-06      100    
end_path:out1:RootOutput                               2.45e-06     5.57671e-06    2.398e-05     5.375e-06    2.77239e-06      100    
end_path:out1:RootOutput(write)                        0.014894      0.253197       1.6547       0.229577      0.236301        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21904.9 MB
  Peak resident set size usage (VmHWM): 2466.67 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_1062_20231203T092006Z_gen_g4_detsim_hitreco__20240508T052611Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_1062_20231203T092006Z_gen_g4_detsim_hitreco__20240508T052611Z_reco2_reco_2024-11-22T_230203Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6410210_1062_20231203T092006Z_gen_g4_detsim_hitreco__20240508T052611Z_reco2_reco_data_2024-11-22T_230203Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work