Jobsub ID 108742.91@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108742.91@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:30:09
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce506
Last heartbeat	2024-11-23 05:08:20
From worker node	Hostname	b9p05p0229.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:38:51
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_216_20231203T041922Z_gen_g4_detsim_hitreco__20240508T062951Z_reco2.root
Jobscript	Exit code	0
	Real time	29m (1751s)
	CPU time	31m (1901s = 108%)
Outputting started	2024-11-23 05:08:02
Output files
Finished	2024-11-23 05:08:20
List job events Wrapper job log
Jobscript log (last 10,000 characters)

1st wire C:0 T:15 P:2 W:731 does not exist (max wire number: 480)
%MSG
334
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 664!
Begin processing the 100th record. run: 6413332 subRun: 1 event: 21700 at 23-Nov-2024 06:07:58 CET
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
> Running Algorithm: Alg0003, LArNeutrinoEventValidation
---RAW-MATCHING-OUTPUT--------------------------------------------------------------------------
Found: 22 with KE=0.00306378 GeV, E=0.00306378 GeV, p2=9.38674e-06
OTHER_INTERACTION (Nuance 1002, Nu 1, CR 0)
IsLost (NNuLosses: 1) 
Parent Neutrino: 16
Neutrino Energy: 0.580527
Neutrino Momentum:   x: -0.121312  y: -0.55511  z: -0.118947 length: 0.580527
PrimaryId 1, Nu 1, CR 0, MCPDG 22, Energy 0.00306378, Dist. 21.0321, nMCHits 20 (6, 12, 2)
-No matched Pfo

------------------------------------------------------------------------------------------------

---INTERPRETED-MATCHING-OUTPUT------------------(my version)-----------------------------------------
Found: 22 with KE=0.00306378 GeV, E=0.00306378 GeV, p2=9.38674e-06
OTHER_INTERACTION (Nuance 1002, Nu 1, CR 0)
IsLost (NNuLosses: 1) 
Parent Neutrino: 16
Neutrino Energy: 0.580527
Neutrino Momentum:   x: -0.121312  y: -0.55511  z: -0.118947 length: 0.580527
PrimaryId 1, Nu 1, CR 0, MCPDG 22, Energy 0.00306378, Dist. 21.0321, nMCHits 20 (6, 12, 2)
-No matched Pfo

---SUMMARY--------------------------------------------------------------------------------------
#CorrectNu: 0/1, Fraction: 0
#Lost: 1 
------------------------------------------------------------------------------------------------

Found: 22 with KE=0.00306378 GeV, E=0.00306378 GeV, p2=9.38674e-06
65
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 77!
23-Nov-2024 06:08:01 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_216_20231203T041922Z_gen_g4_detsim_hitreco__20240508T062951Z_reco2_reco_data_2024-11-23T_043857Z.root"
23-Nov-2024 06:08:01 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/da/ba/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_216_20231203T041922Z_gen_g4_detsim_hitreco__20240508T062951Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2.3643        15.5135       288.24        7.82218       30.8638        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.016498      0.0434147     0.105434      0.0462517     0.0143792       100    
reco:linecluster:LineCluster                           0.146422      0.204661       3.24718      0.162589      0.308368        100    
reco:trajcluster:TrajCluster                           0.0478274     0.208229       7.40324      0.0555404     0.765011        100    
reco:pandora:StandardPandora                           0.796974       1.46555       21.1435       1.19696       2.00668        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000125081    0.0028708     0.0437737    0.00105806    0.00587269       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000165191   0.00852943     0.215195     0.00248106     0.0239377       100    
reco:pandoracalo:Calorimetry                           9.979e-05    0.00841375     0.424955     0.00071705     0.0431275       100    
reco:pandorapid:Chi2ParticleID                        3.3001e-05    0.000136526   0.000629864   0.000112766   9.46931e-05      100    
reco:trkshowersplit:TrackShowerHits                   0.000103181   0.00733054      0.29835     0.000558233    0.0320314       100    
reco:pmtrack:PMAlgTrackMaker                           0.0010162      1.06962       39.3652      0.130381       4.11914        100    
reco:pmtrackcalo:Calorimetry                           6.068e-05    0.00340766     0.0822297    0.000655089   0.00970288       100    
reco:pmtrackpid:Chi2ParticleID                         2.392e-05    0.000184211   0.00144148    0.000115835   0.000204674      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000160621     1.25183       57.3625      0.260458       5.7386         100    
reco:pmtrajfitcalo:Calorimetry                        5.3201e-05    0.00867809      0.41804     0.00104672     0.0424517       100    
reco:pmtrajfitpid:Chi2ParticleID                       2.215e-05     0.0001506    0.000470344   0.000132646   9.32412e-05      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000752845     1.17026       34.3735      0.160036       3.86355        100    
reco:pmtracktccalo:Calorimetry                        5.7491e-05    0.00260774     0.0507435    0.000763235   0.00627669       100    
reco:pmtracktcpid:Chi2ParticleID                       2.297e-05    0.000172315   0.00105774    0.000136201   0.000152094      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000143751    0.725827       16.3841      0.170688       1.81491        100    
reco:pmtrajfittccalo:Calorimetry                       5.199e-05    0.00550507     0.204265     0.000757795    0.0212546       100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.133e-05    0.00023058    0.00213213    0.000134966   0.000274892      100    
reco:blurredcluster:BlurredClustering                  0.0143779     0.737175       5.25519      0.369942      0.919215        100    
reco:emtrkmichelid:EmTrackMichelId                     0.203878       5.67678       96.7391       1.9548        11.653         100    
reco:emshower:EMShower                                0.000246391    0.0198887     0.163018     0.00745783     0.0319371       100    
reco:cvnmap:CVNMapper                                  2.543e-05     0.0111806     0.0305789     0.0132681    0.00915478       100    
reco:cvneva:CVNEvaluator                               1.559e-05     0.612898       1.3018       0.811267      0.431667        100    
reco:energyrecnumu:EnergyReco                         0.000311132   0.00911539     0.106638     0.00407073     0.0160405       100    
reco:energyrecnue:EnergyReco                          0.000121551   0.00106431    0.00989465    0.000457513   0.00160366       100    
reco:energyrecnc:EnergyReco                           0.00165622    0.00332801     0.012889     0.00304217    0.00175634       100    
reco:energyrecnumurange:EnergyReco                    0.000157461   0.00797228     0.139333     0.00285564     0.0180539       100    
reco:energyrecnumumcs:EnergyReco                      0.00118986    0.00901415     0.104608     0.00356795     0.0156245       100    
reco:opdec:Deconvolution                               0.0875054     0.258723      0.420391      0.264639      0.0707124       100    
reco:ophitspe:OpHitFinderDeco                          0.388913       1.7698        3.59593       1.70036      0.763366        100    
reco:opflash:OpFlashFinder                            0.000399863   0.00132518     0.0053134    0.00117011    0.000756252      100    
reco:rns:RandomNumberSaver                             1.686e-05    4.30879e-05   0.000464444   3.6141e-05    5.16533e-05      100    
reco:anglereconue:NuAngularReco                       0.000133601   0.000805838    0.0123132    0.000370973   0.00154222       100    
reco:anglereconumu:NuAngularReco                       6.178e-05    0.000563312    0.0107237    0.000180316   0.00137224       100    
reco:anglereconuepfps:NuAngularReco                   0.000100691   0.000723223    0.0109056    0.000332567   0.00142482       100    
reco:anglereconumupfps:NuAngularReco                  8.2541e-05    0.000620579    0.0107714    0.000233276   0.00140655       100    
[art]:TriggerResults:TriggerResultInserter             8.25e-06     1.30008e-05    7.678e-05     1.193e-05    7.49185e-06      100    
end_path:out1:RootOutput                               2.79e-06     7.00837e-06   3.0311e-05     6.95e-06     3.28721e-06      100    
end_path:out1:RootOutput(write)                        0.0238964     0.202962       0.84018      0.198116      0.136522        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21536.8 MB
  Peak resident set size usage (VmHWM): 2116.01 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_216_20231203T041922Z_gen_g4_detsim_hitreco__20240508T062951Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_216_20231203T041922Z_gen_g4_detsim_hitreco__20240508T062951Z_reco2_reco_2024-11-23T_043857Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6413332_216_20231203T041922Z_gen_g4_detsim_hitreco__20240508T062951Z_reco2_reco_data_2024-11-23T_043857Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work