Jobsub ID 108721.64@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108721.64@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 03:40:08
Site	UK_Lancaster
Entry	UBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat	2024-11-23 06:24:21
From worker node	Hostname	comp23-28
	cpuinfo	Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 03:51:55
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_208_20231120T135745Z_gen_g4_detsim_hitreco__20240503T063056Z_reco2.root
Jobscript	Exit code	0
	Real time	2h (9130s)
	CPU time	2h (8552s = 93%)
Outputting started	2024-11-23 06:24:06
Output files
Finished	2024-11-23 06:24:21
List job events Wrapper job log
Jobscript log (last 10,000 characters)

479092, 
Output 1: 0.95979, 0.0400827, 0.000105519, 2.15057e-05, 
Output 2: 0.996921, 0.0030791, 

455
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1415!
Begin processing the 200th record. run: 6210996 subRun: 1 event: 41800 at 23-Nov-2024 06:24:01 GMT
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
> Running Algorithm: Alg0003, LArNeutrinoEventValidation
---RAW-MATCHING-OUTPUT--------------------------------------------------------------------------
Found: 22 with KE=0.0039656 GeV, E=0.0039656 GeV, p2=1.5726e-05
OTHER_INTERACTION (Nuance 1002, Nu 1, CR 0)
IsLost (NNuLosses: 1) 
Parent Neutrino: 16
Neutrino Energy: 0.219583
Neutrino Momentum:   x: 0.0650081  y: 0.179646  z: 0.108249 length: 0.219583
PrimaryId 1, Nu 1, CR 0, MCPDG 22, Energy 0.0039656, Dist. 14.6036, nMCHits 24 (10, 11, 3)
-No matched Pfo

------------------------------------------------------------------------------------------------

---INTERPRETED-MATCHING-OUTPUT------------------(my version)-----------------------------------------
Found: 22 with KE=0.0039656 GeV, E=0.0039656 GeV, p2=1.5726e-05
OTHER_INTERACTION (Nuance 1002, Nu 1, CR 0)
IsLost (NNuLosses: 1) 
Parent Neutrino: 16
Neutrino Energy: 0.219583
Neutrino Momentum:   x: 0.0650081  y: 0.179646  z: 0.108249 length: 0.219583
PrimaryId 1, Nu 1, CR 0, MCPDG 22, Energy 0.0039656, Dist. 14.6036, nMCHits 24 (10, 11, 3)
-No matched Pfo

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 0/1, Fraction: 0
#Lost: 1 
------------------------------------------------------------------------------------------------

Found: 22 with KE=0.0039656 GeV, E=0.0039656 GeV, p2=1.5726e-05
275
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 552!
23-Nov-2024 06:24:05 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_208_20231120T135745Z_gen_g4_detsim_hitreco__20240503T063056Z_reco2_reco_data_2024-11-23T_035158Z.root"
23-Nov-2024 06:24:05 GMT  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/32/63/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_208_20231120T135745Z_gen_g4_detsim_hitreco__20240503T063056Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.489077       43.2534       2096.79       9.38816       166.483        200    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0123285     0.118668       2.48228      0.0773771      0.21459        200    
reco:linecluster:LineCluster                           0.0659029     0.814262       27.252       0.256572       2.96515        200    
reco:trajcluster:TrajCluster                           0.0309164      2.32339       118.487      0.114658       11.4979        200    
reco:pandora:StandardPandora                           0.0676868      8.38983       1116.38       1.11448       79.5444        200    
reco:pandoraTrack:LArPandoraTrackCreation             0.000129358   0.00985977     0.376103     0.00161036     0.0349185       200    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000154215    0.0099188      0.14922     0.00236367     0.0211314       200    
reco:pandoracalo:Calorimetry                          8.5371e-05     0.047368       3.38929     0.00083832     0.274103        200    
reco:pandorapid:Chi2ParticleID                        3.4261e-05    0.000152364   0.00171716    0.000107855   0.000182807      200    
reco:trkshowersplit:TrackShowerHits                    9.821e-05     0.0362207      1.69052     0.000609875    0.165566        200    
reco:pmtrack:PMAlgTrackMaker                          0.00112647      3.12916       131.821      0.162246       12.105         200    
reco:pmtrackcalo:Calorimetry                          7.2255e-05     0.0198793      1.1508      0.000754311    0.102455        200    
reco:pmtrackpid:Chi2ParticleID                        2.7297e-05    0.000277075   0.00472814    0.000116093   0.000531635      200    
reco:pmtrajfit:PMAlgTrajFitter                        0.000167524     2.38759       74.3925      0.213778       7.89615        200    
reco:pmtrajfitcalo:Calorimetry                        5.9868e-05     0.0476554      3.33088     0.00109897     0.270626        200    
reco:pmtrajfitpid:Chi2ParticleID                       2.525e-05     0.0001601    0.00138168    0.000116384   0.000157131      200    
reco:pmtracktc:PMAlgTrackMaker                        0.000949076     4.1721        184.768      0.182882       16.951         200    
reco:pmtracktccalo:Calorimetry                        6.5241e-05     0.0107936     0.521081     0.000779826    0.0453808       200    
reco:pmtracktcpid:Chi2ParticleID                      2.5976e-05    0.000246066   0.00331228    0.000117011   0.000396219      200    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000153058     1.96232       79.1509       0.20595       6.99245        200    
reco:pmtrajfittccalo:Calorimetry                      5.6731e-05     0.0301393      1.75871     0.000966004    0.147391        200    
reco:pmtrajfittcpid:Chi2ParticleID                     2.413e-05    0.000350616   0.00536186    0.000123577   0.000653429      200    
reco:blurredcluster:BlurredClustering                 5.5788e-05     0.942541       11.7075      0.438629       1.54391        200    
reco:emtrkmichelid:EmTrackMichelId                    8.9117e-05      15.4992       326.282       3.2326        36.8503        200    
reco:emshower:EMShower                                0.000136495    0.0276667     0.321378     0.00488746     0.0526288       200    
reco:cvnmap:CVNMapper                                 1.5604e-05     0.0119059     0.0509983     0.0121962    0.00858868       200    
reco:cvneva:CVNEvaluator                               1.378e-05      0.53754      0.828172      0.706509      0.311431        200    
reco:energyrecnumu:EnergyReco                         0.000250907    0.0873762      2.6404      0.00581629     0.349999        200    
reco:energyrecnue:EnergyReco                          0.000129128   0.00293714     0.0935351    0.000492574   0.00985206       200    
reco:energyrecnc:EnergyReco                           0.00149607    0.00430362     0.0965335    0.00183033     0.0100292       200    
reco:energyrecnumurange:EnergyReco                    0.000171823     0.08445       2.63627     0.00172071     0.349579        200    
reco:energyrecnumumcs:EnergyReco                      0.000702895    0.0859977      2.78025     0.00280784     0.353477        200    
reco:opdec:Deconvolution                               0.0622635     0.345194      0.840339      0.330892      0.136925        200    
reco:ophitspe:OpHitFinderDeco                          0.101043       1.89415       3.75708       1.86614       0.78355        200    
reco:opflash:OpFlashFinder                            0.000186169    0.0011998    0.00374485    0.00111688    0.000536037      200    
reco:rns:RandomNumberSaver                            1.6169e-05    2.44403e-05   0.000263725   1.96215e-05   1.82455e-05      200    
reco:anglereconue:NuAngularReco                       0.000133976   0.00269259     0.101863     0.000329152    0.0102456       200    
reco:anglereconumu:NuAngularReco                      6.3311e-05    0.00251823     0.100127     0.000187885    0.0101057       200    
reco:anglereconuepfps:NuAngularReco                   0.00010394    0.00261655     0.102157     0.000268519    0.010211        200    
reco:anglereconumupfps:NuAngularReco                  8.9512e-05    0.00258954      0.10117     0.000243139    0.0101808       200    
[art]:TriggerResults:TriggerResultInserter             7.691e-06    1.0095e-05    5.6839e-05    9.0725e-06    3.89981e-06      200    
end_path:out1:RootOutput                               1.925e-06    3.72171e-06   1.7703e-05     2.667e-06    2.18677e-06      200    
end_path:out1:RootOutput(write)                       0.00888812     0.206742       1.41898      0.189336      0.175367        200    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9976.62 MB
  Peak resident set size usage (VmHWM): 2825.18 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_208_20231120T135745Z_gen_g4_detsim_hitreco__20240503T063056Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_208_20231120T135745Z_gen_g4_detsim_hitreco__20240503T063056Z_reco2_reco_2024-11-23T_035158Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_208_20231120T135745Z_gen_g4_detsim_hitreco__20240503T063056Z_reco2_reco_data_2024-11-23T_035158Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work