Jobsub ID 108752.15@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108752.15@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:46:20
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce509
Last heartbeat	2024-11-23 06:50:12
From worker node	Hostname	b9g37p7552.cern.ch
	cpuinfo	Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:47:33
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_101_20231208T091547Z_gen_g4_detsim_hitreco__20240510T063232Z_reco2.root
Jobscript	Exit code	0
	Real time	2h (7340s)
	CPU time	1h (7159s = 97%)
Outputting started	2024-11-23 06:49:54
Output files
Finished	2024-11-23 06:50:12
List job events Wrapper job log
Jobscript log (last 10,000 characters)

: 1 event: 10200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:49:41 CET run: 6472782 subRun: 1 event: 10200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:49:41 CET run: 6472782 subRun: 1 event: 10200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:49:41 CET run: 6472782 subRun: 1 event: 10200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:49:41 CET run: 6472782 subRun: 1 event: 10200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:49:41 CET run: 6472782 subRun: 1 event: 10200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:49:41 CET run: 6472782 subRun: 1 event: 10200
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 149, 118, 110
minwire 0: 444
minwire 1: 2016
minwire 2: 305
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.00191423, 0.997791, 0.000294898, 
Output 1: 0.982432, 0.017474, 8.15477e-05, 1.24831e-05, 
Output 2: 0.998998, 0.00100187, 

389
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1260!
23-Nov-2024 07:49:52 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_101_20231208T091547Z_gen_g4_detsim_hitreco__20240510T063232Z_reco2_reco_data_2024-11-23T_044738Z.root"
23-Nov-2024 07:49:52 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/ab/6d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_101_20231208T091547Z_gen_g4_detsim_hitreco__20240510T063232Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2.72969       70.8642       1792.71       14.1835       232.116        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0170987     0.0423763     0.108187      0.0426152     0.0160654       100    
reco:linecluster:LineCluster                           0.150348      0.933348       33.4339      0.187064       4.17121        100    
reco:trajcluster:TrajCluster                           0.0477902      3.42585       124.439      0.0765575      15.8713        100    
reco:pandora:StandardPandora                            1.48558       8.05253       239.292       2.10247       31.8763        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000165209    0.0227023      0.79072     0.00275567     0.0924465       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000224363    0.0256826     0.662633     0.00495827     0.0811558       100    
reco:pandoracalo:Calorimetry                          0.000124574    0.0880916      2.85971     0.00201387     0.391261        100    
reco:pandorapid:Chi2ParticleID                         5.143e-05    0.000269706   0.00304267    0.000179206   0.000369316      100    
reco:trkshowersplit:TrackShowerHits                   0.000131791    0.0658978      2.35944     0.00131833     0.296594        100    
reco:pmtrack:PMAlgTrackMaker                          0.00156474      7.00208       216.686      0.485054       28.1255        100    
reco:pmtrackcalo:Calorimetry                          0.000103269    0.0598266      2.20678     0.00166393     0.285658        100    
reco:pmtrackpid:Chi2ParticleID                         4.331e-05    0.000556049   0.00784797    0.000207067   0.00118767       100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000251453     5.98661       161.692      0.569011       21.5692        100    
reco:pmtrajfitcalo:Calorimetry                        9.7532e-05     0.0979698      3.79462     0.00254244     0.465398        100    
reco:pmtrajfitpid:Chi2ParticleID                      4.2273e-05    0.000294016   0.00196598    0.000199938   0.000307968      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00138378      9.27872       289.247      0.426081       37.9739        100    
reco:pmtracktccalo:Calorimetry                        0.000104574    0.037764       1.43999     0.00167801     0.170002        100    
reco:pmtracktcpid:Chi2ParticleID                      4.4391e-05    0.000506142   0.00647468    0.000215384   0.000959246      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000248888     4.9501        154.247      0.533294       20.2906        100    
reco:pmtrajfittccalo:Calorimetry                      0.000105928    0.078165       3.0514       0.002037      0.390991        100    
reco:pmtrajfittcpid:Chi2ParticleID                    4.5873e-05    0.000656052    0.0103174    0.000213732   0.00141654       100    
reco:blurredcluster:BlurredClustering                  0.0128412      1.85976       23.9513      0.755635       3.39858        100    
reco:emtrkmichelid:EmTrackMichelId                     0.322931       24.8071       506.87        5.27014       67.4907        100    
reco:emshower:EMShower                                0.000311008    0.0648563      1.96508      0.011298      0.204797        100    
reco:cvnmap:CVNMapper                                 3.1816e-05     0.0177119     0.0898708     0.0147995     0.0161781       100    
reco:cvneva:CVNEvaluator                               2.268e-05     0.825035       1.69055      0.957977       0.5093         100    
reco:energyrecnumu:EnergyReco                         0.00035264     0.111692       4.09504      0.0065975     0.448732        100    
reco:energyrecnue:EnergyReco                          0.000207842   0.00939946     0.458026     0.00108742     0.0477032       100    
reco:energyrecnc:EnergyReco                           0.00270247     0.0118976      0.44816     0.00385164     0.0465743       100    
reco:energyrecnumurange:EnergyReco                    0.000225252    0.110747       4.31916     0.00387855     0.468048        100    
reco:energyrecnumumcs:EnergyReco                      0.00276192     0.111186       4.1985      0.00605477     0.457549        100    
reco:opdec:Deconvolution                               0.0972563     0.366504       0.69871      0.358671       0.12394        100    
reco:ophitspe:OpHitFinderDeco                           0.42556       2.01136       4.22899       1.96626      0.787673        100    
reco:opflash:OpFlashFinder                            0.000411531   0.00186588    0.00592721    0.00163993    0.000969424      100    
reco:rns:RandomNumberSaver                            2.1919e-05    4.01225e-05   0.000405281   3.4673e-05    3.83513e-05      100    
reco:anglereconue:NuAngularReco                       0.000176557   0.00686818      0.30092     0.000555093    0.0328787       100    
reco:anglereconumu:NuAngularReco                      0.000104233   0.00667734      0.30608     0.000425906    0.0330465       100    
reco:anglereconuepfps:NuAngularReco                   0.000162106   0.00680124     0.300894     0.000543592    0.0326027       100    
reco:anglereconumupfps:NuAngularReco                  0.000149835   0.00676825     0.301509     0.000529355    0.0326447       100    
[art]:TriggerResults:TriggerResultInserter             1.23e-05     1.86072e-05   7.9728e-05    1.62535e-05   7.94247e-06      100    
end_path:out1:RootOutput                               3.17e-06     6.68713e-06    2.59e-05      5.833e-06    4.07177e-06      100    
end_path:out1:RootOutput(write)                        0.0267668     0.375263       2.74729      0.318339      0.371891        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 10475.1 MB
  Peak resident set size usage (VmHWM): 3139.48 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_101_20231208T091547Z_gen_g4_detsim_hitreco__20240510T063232Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_101_20231208T091547Z_gen_g4_detsim_hitreco__20240510T063232Z_reco2_reco_2024-11-23T_044738Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472782_101_20231208T091547Z_gen_g4_detsim_hitreco__20240510T063232Z_reco2_reco_data_2024-11-23T_044738Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work