Jobsub ID 301707.119@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	301707.119@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:33:23
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce509
Last heartbeat	2024-11-23 05:06:00
From worker node	Hostname	b9p08p1455.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:34:37
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_532_20231202T190907Z_gen_g4_detsim_hitreco__20240508T062819Z_reco2.root
Jobscript	Exit code	0
	Real time	31m (1863s)
	CPU time	33m (2002s = 107%)
Outputting started	2024-11-23 05:05:41
Output files
Finished	2024-11-23 05:06:00
List job events Wrapper job log
Jobscript log (last 10,000 characters)

, CR 0)
IsCorrectNu (NNuMatches: 1) 
Parent Neutrino: 14
Neutrino Energy: 0.361573
Neutrino Momentum:   x: -0.167455  y: 0.310333  z: 0.0799235 length: 0.361574
PrimaryId 1, Nu 1, CR 0, MCPDG 2212, Energy 1.0412, Dist. 8.27507, nMCHits 31 (8, 16, 7)
-MatchedPfoId 1, Nu 1 [NuId: 1], CR 0, PDG 13, nMatchedHits 31 (8, 16, 7), nPfoHits 31 (8, 16, 7)

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 1/1, Fraction: 1
------------------------------------------------------------------------------------------------

Found: 2212 with KE=0.102923 GeV, E=1.0412 GeV, p2=0.203734
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 06:05:31 CET run: 50571149 subRun: 1 event: 53299
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 06:05:32 CET run: 50571149 subRun: 1 event: 53299
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 06:05:32 CET run: 50571149 subRun: 1 event: 53299
Comparing two wires in the same plane: return failure
%MSG
359
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1101!
Begin processing the 100th record. run: 50571149 subRun: 1 event: 53300 at 23-Nov-2024 06:05:35 CET
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
> Running Algorithm: Alg0003, LArNeutrinoEventValidation
---RAW-MATCHING-OUTPUT--------------------------------------------------------------------------
Failure in algorithm Alg0003, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
272
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 636!
23-Nov-2024 06:05:40 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_532_20231202T190907Z_gen_g4_detsim_hitreco__20240508T062819Z_reco2_reco_data_2024-11-23T_043443Z.root"
23-Nov-2024 06:05:40 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/57/d3/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_532_20231202T190907Z_gen_g4_detsim_hitreco__20240508T062819Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.570579       16.7039        232.8        8.15318       32.2618        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0167363     0.0408147     0.0935998     0.0395774     0.0141238       100    
reco:linecluster:LineCluster                           0.0948426      0.25382       5.23615      0.168786      0.525395        100    
reco:trajcluster:TrajCluster                           0.0477596     0.436292       15.8615      0.0557361      1.82404        100    
reco:pandora:StandardPandora                           0.0554969      1.49741       9.84567       1.26946       1.16137        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000119561   0.00351337     0.0527515    0.00111319     0.0077688       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000168552   0.00646005     0.0774217    0.00199406     0.0116031       100    
reco:pandoracalo:Calorimetry                          9.6101e-05    0.00917004     0.356916     0.000789134    0.0387733       100    
reco:pandorapid:Chi2ParticleID                         3.339e-05    0.000150294   0.000772409   0.000122507   0.000104812      100    
reco:trkshowersplit:TrackShowerHits                   8.1961e-05     0.0121728     0.468344     0.000473605    0.053394        100    
reco:pmtrack:PMAlgTrackMaker                          0.000998791     1.23801       37.8216      0.107982       4.46617        100    
reco:pmtrackcalo:Calorimetry                          6.3711e-05    0.00397203     0.086611     0.000657847    0.0122016       100    
reco:pmtrackpid:Chi2ParticleID                         2.47e-05     0.000195593   0.00119481    0.000125761   0.000228441      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000163262     1.12398       20.5673      0.195544       3.14539        100    
reco:pmtrajfitcalo:Calorimetry                        6.3701e-05    0.00951421     0.236914     0.00105999     0.0340221       100    
reco:pmtrajfitpid:Chi2ParticleID                       2.581e-05    0.000161703   0.000698647   0.000132236   0.000129157      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000804709     1.61726       44.8441      0.143756       5.7587         100    
reco:pmtracktccalo:Calorimetry                         5.774e-05    0.00346946     0.0867345    0.000728888    0.0110264       100    
reco:pmtracktcpid:Chi2ParticleID                       2.57e-05     0.000198035   0.00151787    0.000127161   0.000242746      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000146092    0.820668       12.6756      0.169846       2.04966        100    
reco:pmtrajfittccalo:Calorimetry                      5.2991e-05    0.00645839     0.116451     0.00085839     0.0194283       100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.246e-05    0.000248862   0.00192286    0.000135451   0.000326508      100    
reco:blurredcluster:BlurredClustering                 0.000130902    0.751857       6.73826      0.371329       1.05463        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000152592     5.8544        73.6272       1.87699       12.4734        100    
reco:emshower:EMShower                                0.000200082    0.0192374     0.152796     0.00417793     0.0300479       100    
reco:cvnmap:CVNMapper                                 2.6371e-05     0.0102471     0.0390805     0.0137359     0.0086576       100    
reco:cvneva:CVNEvaluator                               1.638e-05     0.585507       1.10376      0.779084       0.43989        100    
reco:energyrecnumu:EnergyReco                         0.000307793    0.0272323      1.2268      0.00343064     0.137877        100    
reco:energyrecnue:EnergyReco                          0.000133681   0.00140313     0.0189044    0.00042308    0.00263734       100    
reco:energyrecnc:EnergyReco                           0.00146808    0.00371434     0.0219738    0.00318512    0.00291316       100    
reco:energyrecnumurange:EnergyReco                    0.000164941    0.0232023      1.13821     0.00261307     0.123501        100    
reco:energyrecnumumcs:EnergyReco                      0.00147944     0.0231105      1.03751     0.00319469     0.111451        100    
reco:opdec:Deconvolution                               0.0524067      0.25662      0.465027      0.261112      0.0843774       100    
reco:ophitspe:OpHitFinderDeco                          0.143265       1.84162       4.18836       1.7162       0.876231        100    
reco:opflash:OpFlashFinder                            0.000151081    0.0014944    0.00671499    0.00138009    0.000844732      100    
reco:rns:RandomNumberSaver                             2.182e-05    4.35255e-05   0.000322453   3.8546e-05    3.15019e-05      100    
reco:anglereconue:NuAngularReco                       0.000179682   0.00103557     0.0186233    0.000388875   0.00234159       100    
reco:anglereconumu:NuAngularReco                      8.3521e-05    0.000763506    0.0172846    0.000195737   0.00217888       100    
reco:anglereconuepfps:NuAngularReco                   0.000122921   0.000937402    0.0173029    0.000342549   0.00219801       100    
reco:anglereconumupfps:NuAngularReco                  0.000105741   0.000829208    0.0173711    0.000264433    0.0021702       100    
[art]:TriggerResults:TriggerResultInserter             9.13e-06     1.53257e-05    7.312e-05     1.454e-05    6.91873e-06      100    
end_path:out1:RootOutput                               2.62e-06     5.90225e-06    2.196e-05     5.63e-06     2.43649e-06      100    
end_path:out1:RootOutput(write)                        0.0129053     0.214662      0.829683      0.219262      0.160187        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21578.9 MB
  Peak resident set size usage (VmHWM): 2159.22 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_532_20231202T190907Z_gen_g4_detsim_hitreco__20240508T062819Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_532_20231202T190907Z_gen_g4_detsim_hitreco__20240508T062819Z_reco2_reco_2024-11-23T_043443Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50571149_532_20231202T190907Z_gen_g4_detsim_hitreco__20240508T062819Z_reco2_reco_data_2024-11-23T_043443Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work