Jobsub ID 301667.81@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	301667.81@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 03:14:21
Site	UK_Brunel
Entry	CMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat	2024-11-23 03:59:44
From worker node	Hostname	wn-a3-37-01.brunel.ac.uk
	cpuinfo	Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 03:18:06
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_633_20231203T112015Z_gen_g4_detsim_hitreco__20240508T045952Z_reco2.root
Jobscript	Exit code	0
	Real time	41m (2486s)
	CPU time	37m (2272s = 91%)
Outputting started	2024-11-23 03:59:32
Output files
Finished	2024-11-23 03:59:44
List job events Wrapper job log
Jobscript log (last 10,000 characters)

6418365 subRun: 1 event: 63399
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 4367, 4456, 2361
minwire 0: 128
minwire 1: 1551
minwire 2: 26
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.00539361, 0.984322, 0.0102847, 
Output 1: 0.214887, 0.349593, 0.214647, 0.220873, 
Output 2: 0.0557495, 0.94425, 

757
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3000!
Begin processing the 100th record. run: 6418365 subRun: 1 event: 63400 at 23-Nov-2024 03:59:26 GMT
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
> Running Algorithm: Alg0003, LArNeutrinoEventValidation
---RAW-MATCHING-OUTPUT--------------------------------------------------------------------------
Failure in algorithm Alg0003, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  23-Nov-2024 03:59:28 GMT run: 6418365 subRun: 1 event: 63400
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 03:59:29 GMT run: 6418365 subRun: 1 event: 63400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 03:59:29 GMT run: 6418365 subRun: 1 event: 63400
Comparing two wires in the same plane: return failure
%MSG
242
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 619!
23-Nov-2024 03:59:31 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_633_20231203T112015Z_gen_g4_detsim_hitreco__20240508T045952Z_reco2_reco_data_2024-11-23T_031808Z.root"
23-Nov-2024 03:59:31 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/6c/50/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_633_20231203T112015Z_gen_g4_detsim_hitreco__20240508T045952Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.20066       21.4762       238.274       8.97726       37.7405        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0249301     0.0546999     0.282973      0.0507712     0.0344631       100    
reco:linecluster:LineCluster                            0.19824      0.346326       4.16433      0.237238      0.474376        100    
reco:trajcluster:TrajCluster                           0.0644513     0.569661       17.1725      0.0782289      2.01635        100    
reco:pandora:StandardPandora                           0.0769494      1.49273       14.3263       1.08855       1.6176         100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000116097   0.00412746     0.0442123    0.00107257    0.00784032       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.00015501    0.00833369      0.14539     0.00174058     0.0171092       100    
reco:pandoracalo:Calorimetry                          9.3016e-05     0.0111177     0.244458     0.000685246    0.035604        100    
reco:pandorapid:Chi2ParticleID                        3.2691e-05    0.000122587   0.00125553    9.2757e-05    0.000133525      100    
reco:trkshowersplit:TrackShowerHits                   9.4209e-05     0.012649      0.256365     0.00060006     0.0404366       100    
reco:pmtrack:PMAlgTrackMaker                          0.00105291      1.28441       21.4902      0.108068       3.45765        100    
reco:pmtrackcalo:Calorimetry                          6.9829e-05    0.00598608     0.097466     0.000596566    0.0155411       100    
reco:pmtrackpid:Chi2ParticleID                        2.7794e-05    0.000192467   0.00140793     9.921e-05    0.000267177      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.00016457      1.6208        50.3716      0.179888       5.54995        100    
reco:pmtrajfitcalo:Calorimetry                        5.6225e-05     0.0124002     0.276641     0.00104325     0.0400869       100    
reco:pmtrajfitpid:Chi2ParticleID                       2.452e-05    0.000141356   0.000981932   0.000112564   0.000127371      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00095676      1.63759       32.2465      0.133401       4.88767        100    
reco:pmtracktccalo:Calorimetry                        6.6115e-05    0.00447414     0.0512075    0.000641013   0.00990951       100    
reco:pmtracktcpid:Chi2ParticleID                      2.6695e-05    0.000182334    0.001095     0.000102305   0.000222526      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000155774     0.86718       8.67332       0.17175       1.71022        100    
reco:pmtrajfittccalo:Calorimetry                       5.692e-05    0.00958401     0.164979     0.000814067    0.0270084       100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.4522e-05    0.000251005   0.00195208    0.000122296   0.000379383      100    
reco:blurredcluster:BlurredClustering                 0.00985871     0.884751       5.31564      0.369822       1.14263        100    
reco:emtrkmichelid:EmTrackMichelId                     0.0597903      9.54048       93.3087       2.74083       17.6415        100    
reco:emshower:EMShower                                0.000222049    0.0158487     0.155816     0.00462168     0.0258278       100    
reco:cvnmap:CVNMapper                                 1.9721e-05     0.0125736     0.0457718     0.0119403     0.0117653       100    
reco:cvneva:CVNEvaluator                              1.5122e-05     0.444473      0.793343      0.651484      0.320106        100    
reco:energyrecnumu:EnergyReco                         0.000253304    0.0319478     0.420822     0.00327351     0.0730804       100    
reco:energyrecnue:EnergyReco                          0.000120462    0.0013769     0.0144825    0.000409396   0.00249687       100    
reco:energyrecnc:EnergyReco                           0.00171407    0.00304509     0.0172164    0.00207592    0.00261402       100    
reco:energyrecnumurange:EnergyReco                    0.000154113    0.029771      0.411757     0.00188222     0.0709235       100    
reco:energyrecnumumcs:EnergyReco                      0.00109815     0.0309677     0.412895     0.00233281     0.0708701       100    
reco:opdec:Deconvolution                               0.0970319     0.322189       1.19204      0.309093      0.128644        100    
reco:ophitspe:OpHitFinderDeco                          0.319017       2.02453       3.54865       2.09582      0.798449        100    
reco:opflash:OpFlashFinder                            0.000191989   0.00113161    0.00319909    0.00110689    0.000527823      100    
reco:rns:RandomNumberSaver                            1.5724e-05    2.56282e-05   0.000245723   2.07175e-05   2.31054e-05      100    
reco:anglereconue:NuAngularReco                       0.000129742   0.00102443     0.0114987    0.000279321   0.00198756       100    
reco:anglereconumu:NuAngularReco                      6.3802e-05    0.00086459     0.0112043    0.000158682   0.00189836       100    
reco:anglereconuepfps:NuAngularReco                   0.000104253   0.000965537    0.0113218    0.000249144   0.00191032       100    
reco:anglereconumupfps:NuAngularReco                  8.8037e-05    0.00091936     0.0113678    0.000204359   0.00190906       100    
[art]:TriggerResults:TriggerResultInserter             8.108e-06    9.97504e-06   4.6163e-05     9.098e-06    3.86094e-06      100    
end_path:out1:RootOutput                               2.135e-06    5.20893e-06   2.9755e-05    4.3485e-06    3.7253e-06       100    
end_path:out1:RootOutput(write)                        0.0121995      0.18492      0.585597      0.196454      0.126349        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 10456 MB
  Peak resident set size usage (VmHWM): 2145.92 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_633_20231203T112015Z_gen_g4_detsim_hitreco__20240508T045952Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_633_20231203T112015Z_gen_g4_detsim_hitreco__20240508T045952Z_reco2_reco_2024-11-23T_031808Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6418365_633_20231203T112015Z_gen_g4_detsim_hitreco__20240508T045952Z_reco2_reco_data_2024-11-23T_031808Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work