Jobsub ID 108723.123@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108723.123@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 03:46:39
Site	UK_Brunel
Entry	CMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat	2024-11-23 05:00:48
From worker node	Hostname	wn-b2-15-00.brunel.ac.uk
	cpuinfo	Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 03:59:26
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_41_20231207T151327Z_gen_g4_detsim_hitreco__20240510T030828Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (3668s)
	CPU time	55m (3314s = 90%)
Outputting started	2024-11-23 05:00:34
Output files
Finished	2024-11-23 05:00:48
List job events Wrapper job log
Jobscript log (last 10,000 characters)

------------------------------------------------------------------------------------------------

---INTERPRETED-MATCHING-OUTPUT------------------(my version)-----------------------------------------
Found: 2212 with KE=0.0336918 GeV, E=0.971964 GeV, p2=0.0643594
NCQEL_P (Nuance 1002, Nu 1, CR 0)
IsCorrectNu (NNuMatches: 1) 
Parent Neutrino: 16
Neutrino Energy: 0.382522
Neutrino Momentum:   x: -0.30882  y: 0.0806931  z: 0.210816 length: 0.382524
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 0.971964, Dist. 1.14834, nMCHits 31 (9, 18, 4)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 13 (5, 4, 4), nPfoHits 13 (5, 4, 4)

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 1/1, Fraction: 1
------------------------------------------------------------------------------------------------

Found: 2212 with KE=0.0336918 GeV, E=0.971964 GeV, p2=0.0643594
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  23-Nov-2024 05:00:30 GMT run: 74568471 subRun: 1 event: 4200
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 05:00:30 GMT run: 74568471 subRun: 1 event: 4200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 05:00:30 GMT run: 74568471 subRun: 1 event: 4200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:00:30 GMT run: 74568471 subRun: 1 event: 4200
Comparing two wires in the same plane: return failure
%MSG
353
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 746!
23-Nov-2024 05:00:33 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_41_20231207T151327Z_gen_g4_detsim_hitreco__20240510T030828Z_reco2_reco_data_2024-11-23T_035929Z.root"
23-Nov-2024 05:00:33 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/ae/f2/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_41_20231207T151327Z_gen_g4_detsim_hitreco__20240510T030828Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2.06395       32.3622       970.34        8.95392       103.05         100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0222028     0.0543398     0.324319       0.05261      0.0338865       100    
reco:linecluster:LineCluster                           0.199462       1.09829       65.737       0.237143       6.59206        100    
reco:trajcluster:TrajCluster                           0.0636692      2.25911       161.303       0.07384       16.1895        100    
reco:pandora:StandardPandora                            1.13388       2.94614       120.877       1.25699       12.0419        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000139018   0.00770844     0.263662     0.00138861     0.0279434       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000179086    0.0137951      0.29393     0.00250114     0.0352925       100    
reco:pandoracalo:Calorimetry                          0.000105499    0.0341059      1.81384     0.000884579    0.187545        100    
reco:pandorapid:Chi2ParticleID                        4.2026e-05    0.000145901   0.00145661    0.000109927   0.000163938      100    
reco:trkshowersplit:TrackShowerHits                   9.2209e-05     0.0384815      2.31941     0.000616357    0.238204        100    
reco:pmtrack:PMAlgTrackMaker                          0.00121133      2.47684       87.2858      0.248441       9.59971        100    
reco:pmtrackcalo:Calorimetry                          7.9199e-05     0.0125775     0.502212     0.000915224    0.0531409       100    
reco:pmtrackpid:Chi2ParticleID                        3.3634e-05    0.000257211   0.00319232    0.000122025   0.000439288      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000195591     2.9373        102.521      0.306936       11.3587        100    
reco:pmtrajfitcalo:Calorimetry                        8.3834e-05     0.0366912      2.01399     0.00106587     0.206822        100    
reco:pmtrajfitpid:Chi2ParticleID                      3.6298e-05    0.000168274   0.00121737    0.000119905   0.000167151      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00107826      3.42105       137.938      0.211428       14.8307        100    
reco:pmtracktccalo:Calorimetry                        8.1689e-05    0.00884331     0.303289     0.000834501    0.0328235       100    
reco:pmtracktcpid:Chi2ParticleID                      3.3215e-05    0.000233769    0.0025526    0.000130311   0.000347601      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000184005     1.6192        44.5739      0.316467       4.96027        100    
reco:pmtrajfittccalo:Calorimetry                      7.9397e-05     0.0230171      1.16565     0.000918861    0.119974        100    
reco:pmtrajfittcpid:Chi2ParticleID                    3.3681e-05    0.000323706   0.00478324    0.00013058    0.000606868      100    
reco:blurredcluster:BlurredClustering                 0.00678292      1.01922       9.90471      0.401855       1.63588        100    
reco:emtrkmichelid:EmTrackMichelId                     0.126183       11.8179       221.767       3.28212       27.1013        100    
reco:emshower:EMShower                                0.000240947    0.0290066     0.350126     0.00716061     0.0556233       100    
reco:cvnmap:CVNMapper                                 2.5332e-05     0.0106864     0.0429877     0.0100199     0.0091285       100    
reco:cvneva:CVNEvaluator                              1.7971e-05     0.495248      0.783893      0.659985      0.310264        100    
reco:energyrecnumu:EnergyReco                         0.000265104    0.0485797      1.4267       0.0032687     0.171441        100    
reco:energyrecnue:EnergyReco                          0.000157395   0.00259766     0.0978784    0.000408277    0.0101588       100    
reco:energyrecnc:EnergyReco                           0.00197056    0.00458798      0.1032      0.00233325     0.0105144       100    
reco:energyrecnumurange:EnergyReco                    0.000156018    0.046161       1.41544     0.00215297     0.169127        100    
reco:energyrecnumumcs:EnergyReco                      0.00204362     0.0470492      1.41647      0.0028253     0.167547        100    
reco:opdec:Deconvolution                               0.0688389     0.329473      0.724251      0.327192      0.108513        100    
reco:ophitspe:OpHitFinderDeco                          0.225941       1.29725       2.5984        1.30732       0.46862        100    
reco:opflash:OpFlashFinder                            0.000144079   0.00123975    0.00392147     0.0011994    0.000598046      100    
reco:rns:RandomNumberSaver                            1.5035e-05    2.55526e-05   0.000231056   2.15315e-05   2.1556e-05       100    
reco:anglereconue:NuAngularReco                       0.000109276   0.00211197     0.0908015    0.000282221   0.00935346       100    
reco:anglereconumu:NuAngularReco                      8.9568e-05    0.00199673     0.0902446    0.000226377   0.00927728       100    
reco:anglereconuepfps:NuAngularReco                   0.000132237   0.00211972     0.0911907    0.000318444   0.00938972       100    
reco:anglereconumupfps:NuAngularReco                  0.000133414   0.00207155     0.0902046    0.000295523   0.00927259       100    
[art]:TriggerResults:TriggerResultInserter             9.807e-06    1.24985e-05   3.5958e-05    1.1456e-05    3.57993e-06      100    
end_path:out1:RootOutput                               2.63e-06     4.68668e-06    1.566e-05     3.744e-06    2.41188e-06      100    
end_path:out1:RootOutput(write)                        0.0117005     0.214759       1.20934      0.212684      0.162828        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8560.56 MB
  Peak resident set size usage (VmHWM): 2469.04 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_41_20231207T151327Z_gen_g4_detsim_hitreco__20240510T030828Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_41_20231207T151327Z_gen_g4_detsim_hitreco__20240510T030828Z_reco2_reco_2024-11-23T_035929Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_41_20231207T151327Z_gen_g4_detsim_hitreco__20240510T030828Z_reco2_reco_data_2024-11-23T_035929Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work