Jobsub ID 108522.164@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108522.164@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-22 22:48:20
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce515
Last heartbeat	2024-11-23 00:13:21
From worker node	Hostname	b9p08p3159.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	stalled
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-22 23:01:39
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006610_922_20231202T210745Z_gen_g4_detsim_hitreco__20240508T070529Z_reco2.root
Outputting started	2024-11-23 00:13:21
Output files
Finished	2024-11-23 00:34:31
List job events Wrapper job log
Jobscript log (last 10,000 characters)

inModule  23-Nov-2024 01:13:15 CET run: 66006610 subRun: 1 event: 92300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:13:15 CET run: 66006610 subRun: 1 event: 92300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:13:15 CET run: 66006610 subRun: 1 event: 92300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:13:15 CET run: 66006610 subRun: 1 event: 92300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:13:15 CET run: 66006610 subRun: 1 event: 92300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:13:15 CET run: 66006610 subRun: 1 event: 92300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 01:13:15 CET run: 66006610 subRun: 1 event: 92300
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 195, 227, 184
minwire 0: 466
minwire 1: 1799
minwire 2: 454
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.308236, 0.585418, 0.106346, 
Output 1: 0.914084, 0.0835142, 0.00225255, 0.000149041, 
Output 2: 0.0833562, 0.916644, 

423
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1255!
23-Nov-2024 01:13:20 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006610_922_20231202T210745Z_gen_g4_detsim_hitreco__20240508T070529Z_reco2_reco_data_2024-11-22T_230146Z.root"
23-Nov-2024 01:13:20 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/ba/10/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006610_922_20231202T210745Z_gen_g4_detsim_hitreco__20240508T070529Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              0.55267       40.6117       916.585       10.5245       123.174        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.016644      0.0447529     0.350958      0.0407589     0.040921        100    
reco:linecluster:LineCluster                           0.0951014     0.773395       37.5943      0.174588       3.96743        100    
reco:trajcluster:TrajCluster                           0.0476304      1.24537       46.2185      0.0581498      6.13916        100    
reco:pandora:StandardPandora                           0.0657436      4.75462       126.547       1.39834       16.9767        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000134941   0.00972343     0.283283     0.00133183     0.0348196       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000172961    0.0172915     0.251759      0.0029764     0.0422871       100    
reco:pandoracalo:Calorimetry                          0.000109461    0.0353092      1.12836     0.00102274     0.162006        100    
reco:pandorapid:Chi2ParticleID                        3.9431e-05    0.000205262   0.00181817    0.000143061   0.000227265      100    
reco:trkshowersplit:TrackShowerHits                   0.000111531    0.0336158     0.954659     0.000686606    0.141683        100    
reco:pmtrack:PMAlgTrackMaker                          0.00118071      4.79132       164.387      0.199866       21.6897        100    
reco:pmtrackcalo:Calorimetry                          7.0981e-05     0.0234179      1.36317     0.000841362    0.139854        100    
reco:pmtrackpid:Chi2ParticleID                         2.692e-05    0.000362704   0.00718243    0.000162806   0.000823438      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000183961     3.37523       121.147      0.265333       13.8742        100    
reco:pmtrajfitcalo:Calorimetry                        6.6101e-05     0.0324648      1.05742     0.00161283     0.142623        100    
reco:pmtrajfitpid:Chi2ParticleID                       2.659e-05    0.000225882    0.0019702    0.000163906   0.000247496      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000994148     5.03629       196.189      0.162365       22.7374        100    
reco:pmtracktccalo:Calorimetry                        6.8021e-05     0.0149592     0.812573     0.000971834    0.0820945       100    
reco:pmtracktcpid:Chi2ParticleID                       2.761e-05    0.000356134    0.0058304    0.000170157   0.000679394      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000168441     2.6875        85.3869      0.264668       10.0141        100    
reco:pmtrajfittccalo:Calorimetry                      7.2911e-05     0.0318353      1.56673     0.00112294     0.168111        100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.965e-05    0.00046909    0.00875827    0.000166422   0.00104266       100    
reco:blurredcluster:BlurredClustering                 0.000175531     1.26851       12.9917      0.521702       2.05922        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000151612     12.6086       258.033       2.85494       33.3275        100    
reco:emshower:EMShower                                0.000243382    0.0429774     0.684477     0.00369299      0.1077         100    
reco:cvnmap:CVNMapper                                  2.744e-05     0.0107575     0.0572219    0.00899003     0.0110273       100    
reco:cvneva:CVNEvaluator                               1.739e-05     0.703296       1.3893        0.99971      0.541051        100    
reco:energyrecnumu:EnergyReco                         0.000327283    0.0494097     0.984931     0.00442781     0.141557        100    
reco:energyrecnue:EnergyReco                          0.000128541   0.00323256     0.0761007    0.000593795   0.00975203       100    
reco:energyrecnc:EnergyReco                           0.00169176    0.00652483     0.102782     0.00388213     0.0123271       100    
reco:energyrecnumurange:EnergyReco                    0.000171101    0.0475472     0.949911     0.00322531     0.141751        100    
reco:energyrecnumumcs:EnergyReco                      0.000977697    0.046559      0.909617     0.00428205     0.131252        100    
reco:opdec:Deconvolution                               0.0525315     0.303977      0.674053      0.300001      0.107195        100    
reco:ophitspe:OpHitFinderDeco                          0.142886       2.32343       4.88482       2.26517       1.00733        100    
reco:opflash:OpFlashFinder                            0.000141701   0.00157051    0.00622058    0.00145599    0.000829918      100    
reco:rns:RandomNumberSaver                            1.9351e-05    5.36295e-05   0.00103325    3.9226e-05    0.000104789      100    
reco:anglereconue:NuAngularReco                       0.000153662   0.00236065     0.0654923    0.000430108   0.00797951       100    
reco:anglereconumu:NuAngularReco                       8.391e-05    0.00223561     0.0720254    0.000241302   0.00864486       100    
reco:anglereconuepfps:NuAngularReco                   0.000155271   0.00230833     0.064409     0.000390184   0.00790106       100    
reco:anglereconumupfps:NuAngularReco                  0.000118581   0.00240792     0.0806735    0.000329327   0.00929156       100    
[art]:TriggerResults:TriggerResultInserter             8.75e-06     1.82337e-05   0.000126771   1.6425e-05    1.24498e-05      100    
end_path:out1:RootOutput                               2.44e-06     6.63985e-06   4.5561e-05     6.78e-06     4.69838e-06      100    
end_path:out1:RootOutput(write)                        0.0163604     0.275049       1.88904      0.279914      0.253299        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21979.1 MB
  Peak resident set size usage (VmHWM): 2519.41 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006610_922_20231202T210745Z_gen_g4_detsim_hitreco__20240508T070529Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006610_922_20231202T210745Z_gen_g4_detsim_hitreco__20240508T070529Z_reco2_reco_2024-11-22T_230146Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006610_922_20231202T210745Z_gen_g4_detsim_hitreco__20240508T070529Z_reco2_reco_data_2024-11-22T_230146Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work