Jobsub ID 301715.57@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	301715.57@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:48:29
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce509
Last heartbeat	2024-11-23 05:44:15
From worker node	Hostname	b9p07p6802.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:50:51
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_95_20231206T033232Z_gen_g4_detsim_hitreco__20240510T062506Z_reco2.root
Jobscript	Exit code	0
	Real time	53m (3184s)
	CPU time	56m (3384s = 106%)
Outputting started	2024-11-23 05:43:56
Output files
Finished	2024-11-23 05:44:15
List job events Wrapper job log
Jobscript log (last 10,000 characters)

rsion)-----------------------------------------
Found: 13 with KE=0.0758294 GeV, E=0.181488 GeV, p2=0.0217741
CCQEL_MU (Nuance 1001, Nu 1, CR 0)
IsCorrectNu (NNuMatches: 1) 
Parent Neutrino: 14
Neutrino Energy: 0.23933
Neutrino Momentum:   x: -0.0277837  y: -0.225801  z: 0.0743047 length: 0.239331
PrimaryId 1, Nu 1, CR 0, MCPDG 13, Energy 0.181488, Dist. 20.5764, nMCHits 65 (15, 35, 15)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 65 (15, 35, 15), nPfoHits 70 (17, 37, 16)

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 1/1, Fraction: 1
------------------------------------------------------------------------------------------------

Found: 13 with KE=0.0758294 GeV, E=0.181488 GeV, p2=0.0217741
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 06:43:52 CET run: 66027687 subRun: 1 event: 9600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 06:43:53 CET run: 66027687 subRun: 1 event: 9600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:43:53 CET run: 66027687 subRun: 1 event: 9600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:43:53 CET run: 66027687 subRun: 1 event: 9600
1st wire C:0 T:20 P:2 W:645 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:43:53 CET run: 66027687 subRun: 1 event: 9600
Comparing two wires in the same plane: return failure
%MSG
271
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 769!
23-Nov-2024 06:43:55 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_95_20231206T033232Z_gen_g4_detsim_hitreco__20240510T062506Z_reco2_reco_data_2024-11-23T_045056Z.root"
23-Nov-2024 06:43:55 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/45/7e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_95_20231206T033232Z_gen_g4_detsim_hitreco__20240510T062506Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2.1555        29.9296       797.027       8.53433       88.9251        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0172273     0.0393925     0.0823086     0.0398327     0.0126526       100    
reco:linecluster:LineCluster                           0.147184      0.496085       9.61226      0.174391       1.44953        100    
reco:trajcluster:TrajCluster                           0.0479528      1.16909       37.6374      0.0651529      5.07947        100    
reco:pandora:StandardPandora                           0.794199       4.65223       291.778       1.16385       28.9896        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000128011   0.00628869     0.182487      0.0014673     0.0200006       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000188791   0.00857761     0.110465     0.00272212     0.016725        100    
reco:pandoracalo:Calorimetry                          0.000100151    0.0320567      1.5274      0.00106671     0.162731        100    
reco:pandorapid:Chi2ParticleID                         3.52e-05     0.000140036   0.000754767   0.000111816   0.000108624      100    
reco:trkshowersplit:TrackShowerHits                   0.000109351    0.0299514      1.01732     0.000671121    0.121146        100    
reco:pmtrack:PMAlgTrackMaker                          0.000980289     2.60144       61.7388      0.174919       8.54957        100    
reco:pmtrackcalo:Calorimetry                           5.904e-05     0.0131345     0.524422     0.000784952    0.0597483       100    
reco:pmtrackpid:Chi2ParticleID                        2.4261e-05    0.000259754   0.00266341    0.000119261   0.000442432      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000153491     2.52627       92.225       0.366152       9.95023        100    
reco:pmtrajfitcalo:Calorimetry                        6.1481e-05     0.0312856      1.47972     0.00139896     0.157096        100    
reco:pmtrajfitpid:Chi2ParticleID                       2.245e-05    0.000164213   0.00124714    0.000118001   0.000156814      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000739847     3.32573       75.8094      0.176374       11.2995        100    
reco:pmtracktccalo:Calorimetry                        5.3511e-05    0.00832611     0.386837     0.000790562    0.0402088       100    
reco:pmtracktcpid:Chi2ParticleID                       2.25e-05     0.000245526   0.00387216    0.000127371   0.000442907      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000162502     1.65419       43.0156      0.275019       5.06715        100    
reco:pmtrajfittccalo:Calorimetry                       5.077e-05     0.0170042      0.67866     0.000955764    0.0745327       100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.252e-05    0.000318562   0.00384066    0.000130521   0.000528552      100    
reco:blurredcluster:BlurredClustering                 0.00883891     0.962094       8.52111      0.504646       1.23442        100    
reco:emtrkmichelid:EmTrackMichelId                     0.144374       9.4813        160.25        2.46099       21.0939        100    
reco:emshower:EMShower                                0.000227902    0.0206655     0.182529     0.00808664     0.0327265       100    
reco:cvnmap:CVNMapper                                  2.282e-05     0.0107779     0.0298014     0.0123089    0.00792589       100    
reco:cvneva:CVNEvaluator                               1.498e-05      0.64575       1.20851      0.730698      0.357215        100    
reco:energyrecnumu:EnergyReco                         0.000321193    0.0386911      1.49393     0.00364927     0.166095        100    
reco:energyrecnue:EnergyReco                          0.000117301   0.00195937     0.0571904    0.000431149   0.00621408       100    
reco:energyrecnc:EnergyReco                           0.00136937    0.00392394     0.0604338    0.00259571    0.00642825       100    
reco:energyrecnumurange:EnergyReco                    0.000151821    0.0386585      1.68306     0.00231609     0.182199        100    
reco:energyrecnumumcs:EnergyReco                      0.000619225    0.0425086       1.954      0.00313264     0.206806        100    
reco:opdec:Deconvolution                               0.131148      0.271825      0.565913      0.268846      0.0759194       100    
reco:ophitspe:OpHitFinderDeco                          0.574569       1.57865       3.59796       1.44596      0.650056        100    
reco:opflash:OpFlashFinder                            0.000392563   0.00125354    0.00299489    0.00111366    0.000530782      100    
reco:rns:RandomNumberSaver                            1.5051e-05    3.2234e-05    0.000260172    3.009e-05    2.44064e-05      100    
reco:anglereconue:NuAngularReco                       0.000141311   0.00167632     0.053438     0.000358693   0.00572812       100    
reco:anglereconumu:NuAngularReco                      6.5501e-05    0.00145285     0.0507741    0.000202027   0.00546691       100    
reco:anglereconuepfps:NuAngularReco                   0.000111751   0.00157961     0.0495792    0.000312628   0.00538866       100    
reco:anglereconumupfps:NuAngularReco                  8.7141e-05    0.00148827     0.0520441    0.000244812   0.00552601       100    
[art]:TriggerResults:TriggerResultInserter             7.74e-06     1.10139e-05   4.6851e-05     9.875e-06    4.77583e-06      100    
end_path:out1:RootOutput                               2.72e-06     5.50097e-06   1.7991e-05     5.29e-06     2.41909e-06      100    
end_path:out1:RootOutput(write)                        0.0287523     0.211582       1.10528      0.184587      0.161371        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21903.4 MB
  Peak resident set size usage (VmHWM): 2486.46 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_95_20231206T033232Z_gen_g4_detsim_hitreco__20240510T062506Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_95_20231206T033232Z_gen_g4_detsim_hitreco__20240510T062506Z_reco2_reco_2024-11-23T_045056Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66027687_95_20231206T033232Z_gen_g4_detsim_hitreco__20240510T062506Z_reco2_reco_data_2024-11-23T_045056Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work