Jobsub ID 108681.64@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108681.64@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 02:01:14
Site	UK_Lancaster
Entry	UBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat	2024-11-23 03:58:50
From worker node	Hostname	comp23-18
	cpuinfo	Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 02:20:20
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6208838_465_20231122T071457Z_gen_g4_detsim_hitreco__20240507T182212Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (5897s)
	CPU time	1h (5224s = 88%)
Outputting started	2024-11-23 03:58:37
Output files
Finished	2024-11-23 03:58:50
List job events Wrapper job log
Jobscript log (last 10,000 characters)

: 0.419565
Neutrino Momentum:   x: -0.0529093  y: -0.404971  z: 0.096092 length: 0.419565
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 1.05977, Dist. 11.101, nMCHits 42 (24, 4, 14)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 38 (24, 0, 14), nPfoHits 38 (24, 0, 14)

------------------------------------------------------------------------------------------------

---INTERPRETED-MATCHING-OUTPUT------------------(my version)-----------------------------------------
Found: 2212 with KE=0.1215 GeV, E=1.05977 GeV, p2=0.242761
NCQEL_P (Nuance 1002, Nu 1, CR 0)
IsCorrectNu (NNuMatches: 1) 
Parent Neutrino: 14
Neutrino Energy: 0.419565
Neutrino Momentum:   x: -0.0529093  y: -0.404971  z: 0.096092 length: 0.419565
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 1.05977, Dist. 11.101, nMCHits 42 (24, 4, 14)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 38 (24, 0, 14), nPfoHits 38 (24, 0, 14)

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 1/1, Fraction: 1
------------------------------------------------------------------------------------------------

Found: 2212 with KE=0.1215 GeV, E=1.05977 GeV, p2=0.242761
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 03:58:32 GMT run: 6208838 subRun: 1 event: 93200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 03:58:32 GMT run: 6208838 subRun: 1 event: 93200
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 03:58:32 GMT run: 6208838 subRun: 1 event: 93200
Comparing two wires in the same plane: return failure
%MSG
351
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 994!
23-Nov-2024 03:58:36 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6208838_465_20231122T071457Z_gen_g4_detsim_hitreco__20240507T182212Z_reco2_reco_data_2024-11-23T_022022Z.root"
23-Nov-2024 03:58:36 GMT  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/be/37/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6208838_465_20231122T071457Z_gen_g4_detsim_hitreco__20240507T182212Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.620047       26.9105       518.711       10.3383       57.5482        200    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0113202     0.121204      0.676011      0.0859488     0.0946775       200    
reco:linecluster:LineCluster                           0.0756327     0.614666       26.703       0.338323       1.98999        200    
reco:trajcluster:TrajCluster                           0.0311081     0.994099       40.582       0.116336       4.51326        200    
reco:pandora:StandardPandora                           0.0705982      2.00297       52.0258        1.349        4.01149        200    
reco:pandoraTrack:LArPandoraTrackCreation             0.000139279   0.00559152     0.103079     0.00165732     0.0124157       200    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000172223    0.0125707     0.364124     0.00268181     0.0403888       200    
reco:pandoracalo:Calorimetry                          9.8523e-05     0.012957      0.358115     0.000891683    0.0458788       200    
reco:pandorapid:Chi2ParticleID                         3.763e-05    0.000147702   0.00117638    0.000117165   0.000132791      200    
reco:trkshowersplit:TrackShowerHits                   7.6275e-05     0.0187627      1.0265      0.000557221    0.0905476       200    
reco:pmtrack:PMAlgTrackMaker                          0.00113821      1.85409       72.7326      0.140336       6.88429        200    
reco:pmtrackcalo:Calorimetry                          7.8598e-05    0.00858263     0.291317     0.000845298    0.0296649       200    
reco:pmtrackpid:Chi2ParticleID                        3.1452e-05    0.000235546   0.00306419    0.000128323   0.000352222      200    
reco:pmtrajfit:PMAlgTrajFitter                        0.000193161     1.5828        48.6981      0.221849       5.24578        200    
reco:pmtrajfitcalo:Calorimetry                        7.1114e-05     0.0140645     0.365703     0.00117349     0.0484842       200    
reco:pmtrajfitpid:Chi2ParticleID                      2.8982e-05    0.000169846   0.000927146   0.000139239   0.000136235      200    
reco:pmtracktc:PMAlgTrackMaker                        0.00112691      2.18672       75.6011      0.179103       8.25305        200    
reco:pmtracktccalo:Calorimetry                        8.0643e-05    0.00622908     0.131842     0.000836791    0.0175082       200    
reco:pmtracktcpid:Chi2ParticleID                      3.0453e-05    0.000219817   0.00206647    0.000132291   0.000277813      200    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000176626     1.23551       29.4205      0.208271       3.37384        200    
reco:pmtrajfittccalo:Calorimetry                      6.6258e-05     0.0133334     0.455673     0.000994572    0.0494739       200    
reco:pmtrajfittcpid:Chi2ParticleID                    2.7166e-05    0.000291198   0.00365169    0.000148025   0.000473485      200    
reco:blurredcluster:BlurredClustering                 7.6636e-05      1.06355       7.90398      0.549842       1.41557        200    
reco:emtrkmichelid:EmTrackMichelId                    0.000127833     11.5834       177.846       3.12995       24.1054        200    
reco:emshower:EMShower                                0.000200886    0.0204774     0.239165     0.00514958     0.0363326       200    
reco:cvnmap:CVNMapper                                 1.9174e-05     0.0132437     0.0530117     0.0133199     0.0103652       200    
reco:cvneva:CVNEvaluator                              1.5408e-05     0.603359       1.02135      0.820512       0.37797        200    
reco:energyrecnumu:EnergyReco                         0.00028754     0.0441563      2.81777     0.00555949     0.223013        200    
reco:energyrecnue:EnergyReco                          0.000142374    0.0017675     0.033818     0.000491963   0.00401422       200    
reco:energyrecnc:EnergyReco                           0.00179931    0.00349565     0.0362103    0.00225187    0.00405183       200    
reco:energyrecnumurange:EnergyReco                    0.000219116    0.0407743      2.77622     0.00211354     0.219485        200    
reco:energyrecnumumcs:EnergyReco                      0.000894926    0.0420906      2.82017     0.00262378     0.221967        200    
reco:opdec:Deconvolution                               0.0750458      0.40063      0.806974      0.384878       0.16353        200    
reco:ophitspe:OpHitFinderDeco                          0.119762       2.19519       4.52914       2.14645      0.937299        200    
reco:opflash:OpFlashFinder                            0.000197317   0.00127268     0.0037499    0.00120108    0.000560721      200    
reco:rns:RandomNumberSaver                            1.8474e-05    2.93761e-05   0.000232908   2.51855e-05   1.6672e-05       200    
reco:anglereconue:NuAngularReco                       0.000158311   0.00131659     0.0281967    0.00033265    0.00324482       200    
reco:anglereconumu:NuAngularReco                      7.1648e-05    0.00111578     0.0271235    0.000184341   0.00310067       200    
reco:anglereconuepfps:NuAngularReco                   0.000119385    0.0012285     0.0272454    0.000280016   0.00313568       200    
reco:anglereconumupfps:NuAngularReco                  9.9678e-05    0.00119167     0.0268351    0.000259305   0.00311397       200    
[art]:TriggerResults:TriggerResultInserter             8.505e-06    1.1857e-05    3.9939e-05    1.0857e-05    3.02813e-06      200    
end_path:out1:RootOutput                               2.551e-06    4.69129e-06   2.1273e-05     3.395e-06    2.63782e-06      200    
end_path:out1:RootOutput(write)                       0.00818106     0.205264      0.938717      0.204852      0.146112        200    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9510.91 MB
  Peak resident set size usage (VmHWM): 2355.13 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6208838_465_20231122T071457Z_gen_g4_detsim_hitreco__20240507T182212Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6208838_465_20231122T071457Z_gen_g4_detsim_hitreco__20240507T182212Z_reco2_reco_2024-11-23T_022022Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6208838_465_20231122T071457Z_gen_g4_detsim_hitreco__20240507T182212Z_reco2_reco_data_2024-11-23T_022022Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work