Jobsub ID 108783.73@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108783.73@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 05:11:36
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce506
Last heartbeat	2024-11-23 06:53:13
From worker node	Hostname	b9p05p5884.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 05:12:44
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_50_20231120T132540Z_gen_g4_detsim_hitreco__20240503T061451Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (6010s)
	CPU time	1h (6007s = 99%)
Outputting started	2024-11-23 06:52:54
Output files
Finished	2024-11-23 06:53:13
List job events Wrapper job log
Jobscript log (last 10,000 characters)

0.173699
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 0.991728, Dist. 2.60027, nMCHits 35 (10, 20, 5)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 15 (7, 3, 5), nPfoHits 15 (7, 3, 5)

------------------------------------------------------------------------------------------------

---INTERPRETED-MATCHING-OUTPUT------------------(my version)-----------------------------------------
Found: 2212 with KE=0.0534559 GeV, E=0.991728 GeV, p2=0.10317
NCQEL_P (Nuance 1002, Nu 1, CR 0)
IsCorrectNu (NNuMatches: 1) 
Parent Neutrino: 12
Neutrino Energy: 0.173698
Neutrino Momentum:   x: 0.0926793  y: -0.0482329  z: 0.138764 length: 0.173699
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 0.991728, Dist. 2.60027, nMCHits 35 (10, 20, 5)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 15 (7, 3, 5), nPfoHits 15 (7, 3, 5)

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 1/1, Fraction: 1
------------------------------------------------------------------------------------------------

Found: 2212 with KE=0.0534559 GeV, E=0.991728 GeV, p2=0.10317
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  23-Nov-2024 07:52:49 CET run: 6207014 subRun: 1 event: 10200
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 07:52:49 CET run: 6207014 subRun: 1 event: 10200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 07:52:49 CET run: 6207014 subRun: 1 event: 10200
Comparing two wires in the same plane: return failure
%MSG
278
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 659!
23-Nov-2024 07:52:53 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_50_20231120T132540Z_gen_g4_detsim_hitreco__20240503T061451Z_reco2_reco_data_2024-11-23T_051249Z.root"
23-Nov-2024 07:52:53 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/3d/c1/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_50_20231120T132540Z_gen_g4_detsim_hitreco__20240503T061451Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.567457       28.352        882.19        8.48246       83.5332        200    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.017203      0.0457094     0.431045      0.041503      0.0340514       200    
reco:linecluster:LineCluster                           0.0944826     0.449318       16.9562      0.171846       1.55462        200    
reco:trajcluster:TrajCluster                           0.0475705      1.07614       60.7613      0.0572668      5.60317        200    
reco:pandora:StandardPandora                           0.0606743      2.37162       92.1628       1.17883       7.59555        200    
reco:pandoraTrack:LArPandoraTrackCreation             0.000109223   0.00581168     0.183718      0.0012728     0.0185829       200    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000148723    0.010425      0.303873     0.00247126     0.0293525       200    
reco:pandoracalo:Calorimetry                          8.9061e-05     0.0312118      2.71027     0.000898194    0.203584        200    
reco:pandorapid:Chi2ParticleID                        4.1021e-05    0.000155837   0.00121359    0.000117712   0.000146894      200    
reco:trkshowersplit:TrackShowerHits                   7.0811e-05     0.0313379      1.22002     0.000596833    0.145765        200    
reco:pmtrack:PMAlgTrackMaker                          0.000895219     2.91626       129.375      0.124798       11.9856        200    
reco:pmtrackcalo:Calorimetry                          5.6971e-05     0.0138116     0.967595     0.000830367    0.0746571       200    
reco:pmtrackpid:Chi2ParticleID                         2.217e-05    0.000272626   0.00350097    0.000138403   0.00050242       200    
reco:pmtrajfit:PMAlgTrajFitter                        0.000150804     2.40152       83.7329      0.230509       8.95335        200    
reco:pmtrajfitcalo:Calorimetry                        5.2161e-05     0.0308186      2.62385     0.00122084     0.195698        200    
reco:pmtrajfitpid:Chi2ParticleID                      2.3001e-05    0.000184919   0.00132398    0.000138518   0.000177446      200    
reco:pmtracktc:PMAlgTrackMaker                        0.000747126     3.64566       154.985      0.167852       14.8391        200    
reco:pmtracktccalo:Calorimetry                        5.3741e-05    0.00980459     0.635147     0.000812132    0.0500889       200    
reco:pmtracktcpid:Chi2ParticleID                       2.18e-05     0.00025684    0.00403965    0.000133152   0.000464272      200    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000137493     1.75028       68.9926      0.198649       6.36906        200    
reco:pmtrajfittccalo:Calorimetry                      5.7141e-05     0.0173276     0.918202     0.000989191    0.0795604       200    
reco:pmtrajfittcpid:Chi2ParticleID                     2.25e-05     0.000340951    0.0038522    0.000146983   0.000650936      200    
reco:blurredcluster:BlurredClustering                 9.3102e-05     0.978667       9.99518      0.425788       1.50742        200    
reco:emtrkmichelid:EmTrackMichelId                    0.000126623     9.24386       234.558       2.09331       24.8364        200    
reco:emshower:EMShower                                0.000201654    0.0312155     0.477703     0.00626444     0.0649872       200    
reco:cvnmap:CVNMapper                                  2.037e-05    0.00892501     0.0544963    0.00698529    0.00840638       200    
reco:cvneva:CVNEvaluator                               1.585e-05     0.701185       6.12692      0.915301       0.59868        200    
reco:energyrecnumu:EnergyReco                         0.000308487    0.0511166      4.49398     0.00403613     0.335875        200    
reco:energyrecnue:EnergyReco                          0.000117283   0.00198077     0.0489127    0.000441554   0.00559363       200    
reco:energyrecnc:EnergyReco                           0.00145975    0.00459599     0.0531072    0.00337862    0.00582665       200    
reco:energyrecnumurange:EnergyReco                    0.000160553    0.0477253      4.28727     0.00295253     0.322445        200    
reco:energyrecnumumcs:EnergyReco                      0.000888859    0.0500151      4.49271     0.00373001     0.335256        200    
reco:opdec:Deconvolution                               0.063092      0.266503      0.960285      0.262939      0.096364        200    
reco:ophitspe:OpHitFinderDeco                           0.14245       1.9192        4.53164       1.8973        0.84393        200    
reco:opflash:OpFlashFinder                            0.000198455    0.0013534    0.00459855    0.00126618    0.000656013      200    
reco:rns:RandomNumberSaver                             1.774e-05    3.96131e-05   0.000409209   3.7266e-05    2.84135e-05      200    
reco:anglereconue:NuAngularReco                       0.000138783   0.00164403     0.0670955    0.000374328   0.00598975       200    
reco:anglereconumu:NuAngularReco                      6.0601e-05    0.00141909     0.0649777    0.000208155    0.005838        200    
reco:anglereconuepfps:NuAngularReco                   0.000100272   0.00157746     0.0668754    0.000356727    0.0059719       200    
reco:anglereconumupfps:NuAngularReco                  8.4432e-05    0.00149352     0.0668759    0.000283831   0.00596457       200    
[art]:TriggerResults:TriggerResultInserter             8.27e-06     1.56256e-05   0.000389728   1.3295e-05    2.68561e-05      200    
end_path:out1:RootOutput                               2.69e-06     7.48509e-06    3.349e-05     7.57e-06     3.29545e-06      200    
end_path:out1:RootOutput(write)                        0.0137667     0.229192       1.49791       0.23577      0.188227        200    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21936.7 MB
  Peak resident set size usage (VmHWM): 2504.32 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_50_20231120T132540Z_gen_g4_detsim_hitreco__20240503T061451Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_50_20231120T132540Z_gen_g4_detsim_hitreco__20240503T061451Z_reco2_reco_2024-11-23T_051249Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6207014_50_20231120T132540Z_gen_g4_detsim_hitreco__20240503T061451Z_reco2_reco_data_2024-11-23T_051249Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work