Jobsub ID 108807.13@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108807.13@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 05:23:22
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce509
Last heartbeat	2024-11-23 07:23:00
From worker node	Hostname	b9g37p7552.cern.ch
	cpuinfo	Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 05:24:59
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_407_20231202T054402Z_gen_g4_detsim_hitreco__20240508T034059Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (7057s)
	CPU time	1h (6850s = 97%)
Outputting started	2024-11-23 07:22:38
Output files
Finished	2024-11-23 07:23:00
List job events Wrapper job log
Jobscript log (last 10,000 characters)

um:   x: 0.517798  y: 0.32306  z: -1.89511 length: 1.99096
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 1.06769, Dist. 12.3657, nMCHits 46 (25, 6, 15)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 46 (25, 6, 15), nPfoHits 46 (25, 6, 15)

------------------------------------------------------------------------------------------------

---INTERPRETED-MATCHING-OUTPUT------------------(my version)-----------------------------------------
Found: 2212 with KE=0.129419 GeV, E=1.06769 GeV, p2=0.259611
NCQEL_P (Nuance 1002, Nu 1, CR 0)
IsCorrectNu (NNuMatches: 1) 
Parent Neutrino: 14
Neutrino Energy: 1.99095
Neutrino Momentum:   x: 0.517798  y: 0.32306  z: -1.89511 length: 1.99096
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 1.06769, Dist. 12.3657, nMCHits 46 (25, 6, 15)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 46 (25, 6, 15), nPfoHits 46 (25, 6, 15)

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 1/1, Fraction: 1
------------------------------------------------------------------------------------------------

Found: 2212 with KE=0.129419 GeV, E=1.06769 GeV, p2=0.259611
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 08:22:26 CET run: 65996932 subRun: 1 event: 40800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 08:22:26 CET run: 65996932 subRun: 1 event: 40800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 08:22:26 CET run: 65996932 subRun: 1 event: 40800
Comparing two wires in the same plane: return failure
%MSG
325
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 843!
23-Nov-2024 08:22:34 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_407_20231202T054402Z_gen_g4_detsim_hitreco__20240508T034059Z_reco2_reco_data_2024-11-23T_052504Z.root"
23-Nov-2024 08:22:34 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/46/8a/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_407_20231202T054402Z_gen_g4_detsim_hitreco__20240508T034059Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.716759       67.7915       1372.77       16.3397       165.712        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0175295     0.0433715      0.27569      0.0435058     0.0255299       100    
reco:linecluster:LineCluster                           0.0957379      1.29772       83.1619      0.196529       8.31961        100    
reco:trajcluster:TrajCluster                           0.0477145      3.5421        178.473      0.0746222      18.8877        100    
reco:pandora:StandardPandora                           0.169682       5.78367       173.148       2.33621       18.3366        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000183495    0.0305464     0.721839     0.00272209     0.0901643       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000242898    0.0479507      1.05623     0.00552259     0.150348        100    
reco:pandoracalo:Calorimetry                          0.000148103    0.0954001      4.10941     0.00176238     0.440257        100    
reco:pandorapid:Chi2ParticleID                        6.0038e-05    0.000327251   0.00176032    0.000210465   0.000331118      100    
reco:trkshowersplit:TrackShowerHits                   0.000220673    0.083129       3.59494     0.00139281     0.387513        100    
reco:pmtrack:PMAlgTrackMaker                          0.00166531      6.58438       158.57       0.317896       20.6042        100    
reco:pmtrackcalo:Calorimetry                          0.000106819    0.0504542      1.7498      0.00169248     0.200964        100    
reco:pmtrackpid:Chi2ParticleID                        4.9005e-05    0.000690821    0.0072105    0.000256025   0.00112234       100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000322117     4.47985       88.2554      0.465256       11.2771        100    
reco:pmtrajfitcalo:Calorimetry                        0.000110009    0.0850895      3.31998     0.00247698     0.365919        100    
reco:pmtrajfitpid:Chi2ParticleID                      4.8124e-05    0.000357033   0.00158803    0.000209811   0.000328638      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00139281      7.19338       191.461      0.530274       22.619         100    
reco:pmtracktccalo:Calorimetry                        0.000117552    0.0311556     0.668783     0.00164357     0.0975526       100    
reco:pmtracktcpid:Chi2ParticleID                       4.634e-05    0.000611767   0.00674381    0.000234213   0.000960438      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000302928     4.13883       106.873      0.537485       12.6226        100    
reco:pmtrajfittccalo:Calorimetry                      0.000105593    0.0588304      1.55901     0.00202716     0.208868        100    
reco:pmtrajfittcpid:Chi2ParticleID                    5.9634e-05    0.000770756    0.008178     0.000267123   0.00136456       100    
reco:blurredcluster:BlurredClustering                 0.000140938     2.07465       15.9555       1.00782       2.88293        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000224644     27.2876       349.373       6.08119       54.1185        100    
reco:emshower:EMShower                                0.000337924    0.0601638     0.695194      0.0148982     0.121233        100    
reco:cvnmap:CVNMapper                                 3.7056e-05     0.026214      0.0738582     0.0263677     0.0178705       100    
reco:cvneva:CVNEvaluator                              2.3882e-05      1.1853        6.34782       1.10225      0.948252        100    
reco:energyrecnumu:EnergyReco                         0.000523021    0.132157       2.73604     0.00831012     0.357257        100    
reco:energyrecnue:EnergyReco                          0.000247317   0.00719428     0.129741     0.000910182    0.0196828       100    
reco:energyrecnc:EnergyReco                            0.0026667     0.0100797     0.122574      0.0041272     0.0183979       100    
reco:energyrecnumurange:EnergyReco                    0.000392269     0.12097       1.98597     0.00396758     0.319752        100    
reco:energyrecnumumcs:EnergyReco                      0.00271316     0.121683       2.34705     0.00508136     0.324862        100    
reco:opdec:Deconvolution                               0.0563079     0.389143      0.742545      0.367262      0.133167        100    
reco:ophitspe:OpHitFinderDeco                          0.196391       2.36146       4.69443       2.26039      0.934269        100    
reco:opflash:OpFlashFinder                            0.000386457   0.00236269    0.00545057    0.00225385    0.00112272       100    
reco:rns:RandomNumberSaver                            2.3043e-05    0.000119174    0.0036989    4.8468e-05    0.000458878      100    
reco:anglereconue:NuAngularReco                       0.000190761   0.00592531     0.114427     0.00066962     0.0169476       100    
reco:anglereconumu:NuAngularReco                      0.00010911     0.0054296     0.101817     0.000432378    0.0158267       100    
reco:anglereconuepfps:NuAngularReco                   0.000166658   0.00576253     0.101055     0.000627316    0.0160055       100    
reco:anglereconumupfps:NuAngularReco                  0.000153581   0.00586416     0.115834     0.000547895    0.016976        100    
[art]:TriggerResults:TriggerResultInserter            1.3371e-05    2.41285e-05   6.7166e-05    2.28355e-05   7.8671e-06       100    
end_path:out1:RootOutput                               3.156e-06    8.60476e-06   2.5211e-05    8.4555e-06    4.30905e-06      100    
end_path:out1:RootOutput(write)                        0.0213106     0.437489       1.76622      0.390334      0.309708        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9855.17 MB
  Peak resident set size usage (VmHWM): 2644.93 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_407_20231202T054402Z_gen_g4_detsim_hitreco__20240508T034059Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_407_20231202T054402Z_gen_g4_detsim_hitreco__20240508T034059Z_reco2_reco_2024-11-23T_052504Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_65996932_407_20231202T054402Z_gen_g4_detsim_hitreco__20240508T034059Z_reco2_reco_data_2024-11-23T_052504Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work