Jobsub ID 108743.29@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108743.29@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:34:30
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce511
Last heartbeat	2024-11-23 05:14:17
From worker node	Hostname	b9p07p1853.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:41:26
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_254_20231203T033452Z_gen_g4_detsim_hitreco__20240508T071607Z_reco2.root
Jobscript	Exit code	0
	Real time	32m (1955s)
	CPU time	36m (2200s = 112%)
Outputting started	2024-11-23 05:14:01
Output files
Finished	2024-11-23 05:14:17
List job events Wrapper job log
Jobscript log (last 10,000 characters)

he same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:13:57 CET run: 66006611 subRun: 1 event: 25500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:13:58 CET run: 66006611 subRun: 1 event: 25500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:13:58 CET run: 66006611 subRun: 1 event: 25500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:13:58 CET run: 66006611 subRun: 1 event: 25500
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:13:58 CET run: 66006611 subRun: 1 event: 25500
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 62, 38, 38
minwire 0: 530
minwire 1: 2056
minwire 2: 517
Used alternate method to get min and max wires due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0480257, 0.853959, 0.0980152, 
Output 1: 0.828612, 0.169428, 0.00179118, 0.000168606, 
Output 2: 0.992399, 0.00760104, 

252
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 665!
23-Nov-2024 06:14:00 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_254_20231203T033452Z_gen_g4_detsim_hitreco__20240508T071607Z_reco2_reco_data_2024-11-23T_044131Z.root"
23-Nov-2024 06:14:00 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/27/a0/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_254_20231203T033452Z_gen_g4_detsim_hitreco__20240508T071607Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.42503       17.629        224.942       6.99271       31.758         100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0167065     0.0394954     0.0748417     0.0405346     0.0119385       100    
reco:linecluster:LineCluster                           0.0938661     0.229291       1.74847       0.16574      0.258502        100    
reco:trajcluster:TrajCluster                           0.0470209     0.379381       8.41313      0.0551584      1.10579        100    
reco:pandora:StandardPandora                           0.0610322      1.46135       17.065        1.01756       1.83542        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000107662    0.0036469     0.0435044    0.000961013   0.00714822       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000161162   0.00848696     0.161733     0.00196081     0.0205946       100    
reco:pandoracalo:Calorimetry                          9.1361e-05     0.0112637     0.335632     0.000650959    0.0412416       100    
reco:pandorapid:Chi2ParticleID                        3.4141e-05    0.000132704   0.00143574    9.8342e-05    0.000152062      100    
reco:trkshowersplit:TrackShowerHits                   8.6292e-05     0.0125016     0.280131     0.000415271    0.0405691       100    
reco:pmtrack:PMAlgTrackMaker                          0.000941613     1.26466       21.5571      0.125728       3.20537        100    
reco:pmtrackcalo:Calorimetry                          5.8751e-05    0.00518657     0.115507     0.000546283    0.0158069       100    
reco:pmtrackpid:Chi2ParticleID                         2.352e-05    0.000193069   0.00145593    0.000116866   0.000245344      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000141052     1.32423       28.7053      0.161659       3.67487        100    
reco:pmtrajfitcalo:Calorimetry                        5.4101e-05     0.0110079      0.34688     0.000845072    0.0395239       100    
reco:pmtrajfitpid:Chi2ParticleID                       2.235e-05    0.000146819   0.000563918   0.000126066   0.000105531      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000753861     1.61542       35.3071      0.117278       4.60095        100    
reco:pmtracktccalo:Calorimetry                        5.4901e-05    0.00443099     0.0824213    0.00065998     0.0127177       100    
reco:pmtracktcpid:Chi2ParticleID                       2.305e-05    0.000196556   0.00153731    0.000126367   0.00024901       100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000123241     1.03665       20.1416      0.165554       2.67197        100    
reco:pmtrajfittccalo:Calorimetry                      5.5941e-05     0.0105108     0.340745     0.000783946    0.0392988       100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.245e-05    0.000268249   0.00381145    0.000136942   0.000490521      100    
reco:blurredcluster:BlurredClustering                 0.000162602    0.831413       5.70061      0.504476       1.04959        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000240693     6.8721        80.0175       1.66492       13.4209        100    
reco:emshower:EMShower                                0.000232403    0.0228306      0.29922     0.00330675     0.0428339       100    
reco:cvnmap:CVNMapper                                  2.141e-05    0.00773521     0.0304946    0.00573358    0.00751667       100    
reco:cvneva:CVNEvaluator                               1.549e-05     0.522092       1.06392      0.725148      0.381206        100    
reco:energyrecnumu:EnergyReco                         0.000292404    0.019988       0.44442     0.00306687     0.0568023       100    
reco:energyrecnue:EnergyReco                          0.000115232   0.00129294     0.0134318    0.000363725   0.00228301       100    
reco:energyrecnc:EnergyReco                           0.00142615    0.00299694     0.0172853    0.00206802    0.00261524       100    
reco:energyrecnumurange:EnergyReco                    0.000130582    0.0191728     0.427046     0.00177783     0.0581801       100    
reco:energyrecnumumcs:EnergyReco                      0.00136677     0.020806      0.522858     0.00244353     0.0642449       100    
reco:opdec:Deconvolution                               0.100718      0.253826      0.454218       0.24732      0.0766523       100    
reco:ophitspe:OpHitFinderDeco                          0.325428       1.45353       3.22769       1.37734      0.601628        100    
reco:opflash:OpFlashFinder                            0.000263383   0.00112198    0.00427245    0.00101203    0.000573074      100    
reco:rns:RandomNumberSaver                             1.846e-05    3.48625e-05   0.000250104   3.3175e-05    2.46418e-05      100    
reco:anglereconue:NuAngularReco                       0.000148772   0.00093239      0.01175     0.00033112    0.00179548       100    
reco:anglereconumu:NuAngularReco                      6.4371e-05    0.000741753    0.0109954    0.000184852   0.00167324       100    
reco:anglereconuepfps:NuAngularReco                   0.000111121   0.000866842    0.0109924    0.00031392    0.00170129       100    
reco:anglereconumupfps:NuAngularReco                  9.0071e-05    0.000786312    0.0108927    0.000240293   0.00164392       100    
[art]:TriggerResults:TriggerResultInserter             8.34e-06     1.08479e-05   3.6301e-05     9.855e-06    3.66262e-06      100    
end_path:out1:RootOutput                               3.16e-06     6.79342e-06   3.2041e-05    6.6205e-06    4.15801e-06      100    
end_path:out1:RootOutput(write)                        0.0182924     0.176593      0.591584      0.175699      0.124104        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21683.5 MB
  Peak resident set size usage (VmHWM): 2248.16 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_254_20231203T033452Z_gen_g4_detsim_hitreco__20240508T071607Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_254_20231203T033452Z_gen_g4_detsim_hitreco__20240508T071607Z_reco2_reco_2024-11-23T_044131Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66006611_254_20231203T033452Z_gen_g4_detsim_hitreco__20240508T071607Z_reco2_reco_data_2024-11-23T_044131Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work