Jobsub ID 301703.11@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	301703.11@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:29:03
Site	UK_Durham
Entry	DUNE_UK_SGridDurham_ce3
Last heartbeat	2024-11-23 05:27:06
From worker node	Hostname	n259.dur.scotgrid.ac.uk
	cpuinfo	AMD EPYC 7763 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:30:01
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492179_216_20231208T035214Z_gen_g4_detsim_hitreco__20240510T034051Z_reco2.root
Jobscript	Exit code	0
	Real time	56m (3408s)
	CPU time	50m (3052s = 89%)
Outputting started	2024-11-23 05:26:49
Output files
Finished	2024-11-23 05:27:06
List job events Wrapper job log
Jobscript log (last 10,000 characters)

e same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 05:26:40 GMT run: 6492179 subRun: 1 event: 21700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 05:26:40 GMT run: 6492179 subRun: 1 event: 21700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:26:40 GMT run: 6492179 subRun: 1 event: 21700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:26:40 GMT run: 6492179 subRun: 1 event: 21700
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:26:40 GMT run: 6492179 subRun: 1 event: 21700
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 43, 47, 39
minwire 0: 2578
minwire 1: 664
minwire 2: 2416
Used alternate method to get min and max wires due to vertex determination failure: 57, 256
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 15, 214
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 111, 310
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.987173, 0.00263997, 0.0101872, 
Output 1: 0.872054, 0.113345, 0.0119413, 0.00266007, 
Output 2: 0.993086, 0.00691453, 

354
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 944!
23-Nov-2024 05:26:48 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492179_216_20231208T035214Z_gen_g4_detsim_hitreco__20240510T034051Z_reco2_reco_data_2024-11-23T_043006Z.root"
23-Nov-2024 05:26:48 GMT  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/09/e5/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492179_216_20231208T035214Z_gen_g4_detsim_hitreco__20240510T034051Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.650176       30.3295       298.587       10.7126       56.9041        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0171796     0.0434405     0.380796      0.0394035     0.0441929       100    
reco:linecluster:LineCluster                           0.0997491     0.372544       4.09706      0.188543      0.569562        100    
reco:trajcluster:TrajCluster                           0.0479447     0.671735       12.1184      0.0631266      2.1685         100    
reco:pandora:StandardPandora                           0.0642921      2.08911       21.534        1.24178       3.20568        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000164291   0.00678697     0.0833533     0.001778      0.0153225       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000212543    0.012112      0.135306      0.0037995     0.0236094       100    
reco:pandoracalo:Calorimetry                          0.000123834    0.0225556     0.397342     0.00157309     0.0660989       100    
reco:pandorapid:Chi2ParticleID                        4.3152e-05    0.000200157   0.00236455    0.000154042   0.000251837      100    
reco:trkshowersplit:TrackShowerHits                   0.000114597    0.0262851     0.562719     0.000962174    0.0883232       100    
reco:pmtrack:PMAlgTrackMaker                          0.00139749      2.54107       36.541       0.236456       6.78306        100    
reco:pmtrackcalo:Calorimetry                          8.0452e-05     0.0122497     0.244166     0.00118717     0.0380464       100    
reco:pmtrackpid:Chi2ParticleID                        3.0367e-05    0.000345001   0.00246369    0.000162623   0.000487942      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000243532     2.34164       41.0703      0.384792       6.01418        100    
reco:pmtrajfitcalo:Calorimetry                        7.8428e-05     0.0231813     0.405436     0.00199206     0.0671412       100    
reco:pmtrajfitpid:Chi2ParticleID                       3.154e-05    0.000227596    0.0010292    0.000175974   0.000204588      100    
reco:pmtracktc:PMAlgTrackMaker                         0.0011218      3.24212       49.2876      0.235156       8.83843        100    
reco:pmtracktccalo:Calorimetry                        8.0292e-05    0.00820925      0.13973     0.00119851     0.022651        100    
reco:pmtracktcpid:Chi2ParticleID                      3.2682e-05    0.000315539   0.00208792    0.000178007   0.000385642      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000183427     1.85823       23.2953      0.320244       4.14857        100    
reco:pmtrajfittccalo:Calorimetry                      7.5253e-05     0.0169324     0.321954      0.0013831     0.0506159       100    
reco:pmtrajfittcpid:Chi2ParticleID                    3.3423e-05    0.000443351   0.00340246    0.000204884   0.000639556      100    
reco:blurredcluster:BlurredClustering                 9.0321e-05      1.06583       7.21834      0.649067       1.28458        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000156988     12.372        117.076       3.6637        23.3836        100    
reco:emshower:EMShower                                0.000183407     0.03162      0.449361      0.0108714     0.064718        100    
reco:cvnmap:CVNMapper                                 2.1621e-05     0.0124336     0.0444073     0.0141634    0.00915231       100    
reco:cvneva:CVNEvaluator                              2.0299e-05      0.65679      0.936967      0.818233      0.340005        100    
reco:energyrecnumu:EnergyReco                         0.000426549    0.0656075      2.50344     0.00574665     0.306643        100    
reco:energyrecnue:EnergyReco                          0.000162097   0.00220774     0.0298067    0.000643506   0.00482145       100    
reco:energyrecnc:EnergyReco                           0.00256587    0.00471573     0.0335115    0.00308945    0.00512876       100    
reco:energyrecnumurange:EnergyReco                    0.000197114    0.0629571      2.49488     0.00281774     0.306106        100    
reco:energyrecnumumcs:EnergyReco                      0.000913502    0.0647189      2.51674      0.0036394     0.307646        100    
reco:opdec:Deconvolution                               0.0583638     0.341102       1.69459      0.283596      0.224746        100    
reco:ophitspe:OpHitFinderDeco                          0.232472       2.08367       3.61013       2.05614      0.745942        100    
reco:opflash:OpFlashFinder                            0.000291463   0.00165661    0.00778791    0.00150498    0.000877735      100    
reco:rns:RandomNumberSaver                            2.3164e-05    4.82758e-05   0.000455303   4.4524e-05    4.24755e-05      100    
reco:anglereconue:NuAngularReco                       0.000192154   0.00169931     0.025068     0.000435982   0.00407396       100    
reco:anglereconumu:NuAngularReco                      8.3398e-05    0.00142889     0.024325     0.00026369     0.0039188       100    
reco:anglereconuepfps:NuAngularReco                   0.000146527   0.00158841     0.0231366    0.000403019   0.00387171       100    
reco:anglereconumupfps:NuAngularReco                  0.000126319   0.00149097     0.0238676    0.000333062   0.00388219       100    
[art]:TriggerResults:TriggerResultInserter             1.097e-05    1.58133e-05   5.8461e-05    1.4628e-05    5.52385e-06      100    
end_path:out1:RootOutput                               3.487e-06    8.37298e-06   2.4376e-05    8.2155e-06    3.17083e-06      100    
end_path:out1:RootOutput(write)                        0.0138961     0.265893      0.870187       0.25163      0.166883        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 38393.1 MB
  Peak resident set size usage (VmHWM): 2336.74 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492179_216_20231208T035214Z_gen_g4_detsim_hitreco__20240510T034051Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492179_216_20231208T035214Z_gen_g4_detsim_hitreco__20240510T034051Z_reco2_reco_2024-11-23T_043006Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6492179_216_20231208T035214Z_gen_g4_detsim_hitreco__20240510T034051Z_reco2_reco_data_2024-11-23T_043006Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work