Jobsub ID 108883.99@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108883.99@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 07:26:49
Site	NL_SURFsara
Entry	DUNE_SurfSARA_arc03
Last heartbeat	2024-11-23 08:15:23
From worker node	Hostname	wn-da-57.gina.surfsara.nl
	cpuinfo	AMD EPYC 7702P 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	129600 (36 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 07:29:12
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66050399_19_20231207T162308Z_gen_g4_detsim_hitreco__20240510T041537Z_reco2.root
Jobscript	Exit code	0
	Real time	45m (2758s)
	CPU time	43m (2606s = 94%)
Outputting started	2024-11-23 08:15:11
Output files
Finished	2024-11-23 08:15:23
List job events Wrapper job log
Jobscript log (last 10,000 characters)

omentum:   x: 0.0314752  y: -0.328959  z: 0.123483 length: 0.352779
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 1.03244, Dist. 7.16179, nMCHits 29 (15, 6, 8)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 29 (15, 6, 8), nPfoHits 29 (15, 6, 8)

------------------------------------------------------------------------------------------------

---INTERPRETED-MATCHING-OUTPUT------------------(my version)-----------------------------------------
Found: 2212 with KE=0.0941659 GeV, E=1.03244 GeV, p2=0.185574
NCQEL_P (Nuance 1002, Nu 1, CR 0)
IsCorrectNu (NNuMatches: 1) 
Parent Neutrino: 12
Neutrino Energy: 0.352778
Neutrino Momentum:   x: 0.0314752  y: -0.328959  z: 0.123483 length: 0.352779
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 1.03244, Dist. 7.16179, nMCHits 29 (15, 6, 8)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 29 (15, 6, 8), nPfoHits 29 (15, 6, 8)

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 1/1, Fraction: 1
------------------------------------------------------------------------------------------------

Found: 2212 with KE=0.0941659 GeV, E=1.03244 GeV, p2=0.185574
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 09:15:07 CET run: 66050399 subRun: 1 event: 2000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 09:15:07 CET run: 66050399 subRun: 1 event: 2000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 09:15:07 CET run: 66050399 subRun: 1 event: 2000
Comparing two wires in the same plane: return failure
%MSG
301
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 694!
23-Nov-2024 09:15:09 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66050399_19_20231207T162308Z_gen_g4_detsim_hitreco__20240510T041537Z_reco2_reco_data_2024-11-23T_072916Z.root"
23-Nov-2024 09:15:09 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/1c/5c/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66050399_19_20231207T162308Z_gen_g4_detsim_hitreco__20240510T041537Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.489887       25.686        272.651       9.76989       42.872         100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0159166     0.0371715     0.068659      0.0397247     0.009562        100    
reco:linecluster:LineCluster                           0.0867317      0.25729       7.47529      0.161747      0.736079        100    
reco:trajcluster:TrajCluster                           0.0432462     0.506874       20.3097      0.053626       2.19083        100    
reco:pandora:StandardPandora                           0.0517339      1.62725       19.1548       1.15688       2.16782        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000131681   0.00513721     0.0465027    0.00157849    0.00912706       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000176226    0.0138878     0.258777     0.00305456     0.0351796       100    
reco:pandoracalo:Calorimetry                          0.000102275    0.0125157     0.339857     0.00110289     0.0408214       100    
reco:pandorapid:Chi2ParticleID                        3.9415e-05    0.00013573    0.000582239   0.000105992   9.07087e-05      100    
reco:trkshowersplit:TrackShowerHits                   6.1357e-05     0.0129866     0.350747     0.000820938    0.0435084       100    
reco:pmtrack:PMAlgTrackMaker                          0.00124273      1.64039       28.1185      0.243661       4.22606        100    
reco:pmtrackcalo:Calorimetry                          7.1827e-05    0.00800687     0.174995     0.00103776     0.0221636       100    
reco:pmtrackpid:Chi2ParticleID                        2.7472e-05    0.000251984   0.00225196    0.000148602   0.000327214      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000196374     1.37519       22.5384      0.368599       3.0396         100    
reco:pmtrajfitcalo:Calorimetry                        6.5345e-05     0.0129508     0.325718     0.00159358     0.039922        100    
reco:pmtrajfitpid:Chi2ParticleID                      2.6401e-05    0.00017144    0.000905736   0.000141915   0.000138113      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00106872      2.03669       34.6059      0.181889       5.31818        100    
reco:pmtracktccalo:Calorimetry                        6.9963e-05    0.00650707     0.0842108    0.00107479      0.01555        100    
reco:pmtracktcpid:Chi2ParticleID                      2.7252e-05    0.000229655   0.00168188    0.000146659   0.000253384      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000167699     1.3389        22.8793      0.269862       2.92038        100    
reco:pmtrajfittccalo:Calorimetry                      6.3952e-05     0.011419      0.322367     0.00111014     0.0375924       100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.617e-05    0.000308055   0.00305432    0.000153322   0.000434601      100    
reco:blurredcluster:BlurredClustering                 6.0916e-05      1.2072        7.68392      0.576716       1.57101        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000103337     13.2012       131.842       4.29338       22.0697        100    
reco:emshower:EMShower                                0.000153253    0.0206983     0.182045     0.00380182     0.0307924       100    
reco:cvnmap:CVNMapper                                 1.7594e-05    0.00861035     0.0427374    0.00977889    0.00578177       100    
reco:cvneva:CVNEvaluator                               1.564e-05     0.548965      0.867078      0.690532       0.29294        100    
reco:energyrecnumu:EnergyReco                         0.000309349    0.0247227     0.751534     0.00471317     0.0819291       100    
reco:energyrecnue:EnergyReco                          0.000140958   0.00163076     0.0149108    0.000560377   0.00276547       100    
reco:energyrecnc:EnergyReco                           0.00202346    0.00368715     0.0182224    0.00256329    0.00293116       100    
reco:energyrecnumurange:EnergyReco                    0.000159332    0.0227456     0.752533     0.00241207     0.0819414       100    
reco:energyrecnumumcs:EnergyReco                      0.00146536     0.0240248     0.748641     0.00328472     0.0815245       100    
reco:opdec:Deconvolution                               0.0467049     0.271398      0.469608      0.276806      0.0856771       100    
reco:ophitspe:OpHitFinderDeco                           0.14417       1.22867       2.26349       1.25823      0.465636        100    
reco:opflash:OpFlashFinder                            0.000177508   0.00121851    0.00343634    0.00121031    0.000489939      100    
reco:rns:RandomNumberSaver                            1.8845e-05    2.56865e-05   0.000297316   2.1792e-05    2.75932e-05      100    
reco:anglereconue:NuAngularReco                       0.000159825   0.00116287     0.0113828    0.00035976    0.00232427       100    
reco:anglereconumu:NuAngularReco                      6.8741e-05     0.0010009     0.0107466    0.000217354   0.00225258       100    
reco:anglereconuepfps:NuAngularReco                   0.00012583    0.00114166     0.0110893    0.000357501   0.00230644       100    
reco:anglereconumupfps:NuAngularReco                  9.6163e-05    0.00106968     0.0107574    0.000280579   0.00227124       100    
[art]:TriggerResults:TriggerResultInserter             8.766e-06    1.01693e-05   3.7621e-05     9.718e-06    2.87201e-06      100    
end_path:out1:RootOutput                               2.895e-06    3.96643e-06   1.8946e-05     3.577e-06    2.16881e-06      100    
end_path:out1:RootOutput(write)                       0.00782466     0.210845      0.730136      0.203903      0.130749        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 12396.7 MB
  Peak resident set size usage (VmHWM): 2134.98 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66050399_19_20231207T162308Z_gen_g4_detsim_hitreco__20240510T041537Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66050399_19_20231207T162308Z_gen_g4_detsim_hitreco__20240510T041537Z_reco2_reco_2024-11-23T_072916Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_66050399_19_20231207T162308Z_gen_g4_detsim_hitreco__20240510T041537Z_reco2_reco_data_2024-11-23T_072916Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work