Jobsub ID 108724.7@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108724.7@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 03:47:42
Site	UK_Brunel
Entry	CMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat	2024-11-23 05:36:20
From worker node	Hostname	wn-b8-26-01.brunel.ac.uk
	cpuinfo	Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:00:50
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_1198_20231203T160707Z_gen_g4_detsim_hitreco__20240508T072521Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (5716s)
	CPU time	1h (5383s = 94%)
Outputting started	2024-11-23 05:36:06
Output files
Finished	2024-11-23 05:36:20
List job events Wrapper job log
Jobscript log (last 10,000 characters)

Run: 1 event: 119900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 05:35:58 GMT run: 6414162 subRun: 1 event: 119900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 05:35:58 GMT run: 6414162 subRun: 1 event: 119900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 05:35:58 GMT run: 6414162 subRun: 1 event: 119900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 05:35:58 GMT run: 6414162 subRun: 1 event: 119900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:35:58 GMT run: 6414162 subRun: 1 event: 119900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 05:35:58 GMT run: 6414162 subRun: 1 event: 119900
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 86, 283, 149
minwire 0: 191
minwire 1: 2230
minwire 2: 116
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0231434, 0.000506271, 0.97635, 
Output 1: 0.783303, 0.206769, 0.00913386, 0.000794907, 
Output 2: 0.980789, 0.019211, 

343
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 905!
23-Nov-2024 05:36:05 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_1198_20231203T160707Z_gen_g4_detsim_hitreco__20240508T072521Z_reco2_reco_data_2024-11-23T_040053Z.root"
23-Nov-2024 05:36:05 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/74/d6/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_1198_20231203T160707Z_gen_g4_detsim_hitreco__20240508T072521Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.705679       53.2553       3475.33       8.24799       345.155        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                  0.02294      0.0569329     0.718909      0.052506      0.0687095       100    
reco:linecluster:LineCluster                           0.126956      0.782127       49.1321      0.237913       4.86387        100    
reco:trajcluster:TrajCluster                           0.0637221      3.29411       289.411      0.0753739      28.7978        100    
reco:pandora:StandardPandora                           0.0726567      22.548        2068.13       1.32243       205.614        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000128922   0.00880901     0.516494      0.0012406     0.0514174       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.00018004     0.014428      0.299005     0.00268479     0.0388143       100    
reco:pandoracalo:Calorimetry                          0.000103023    0.0364076      2.65303     0.000806685    0.265011        100    
reco:pandorapid:Chi2ParticleID                        4.1541e-05    0.000139406   0.00111515    0.000109637   0.000133149      100    
reco:trkshowersplit:TrackShowerHits                   9.1864e-05     0.0340851      2.61623     0.000558165    0.261376        100    
reco:pmtrack:PMAlgTrackMaker                          0.00121996      3.12394       204.818      0.208118       20.4092        100    
reco:pmtrackcalo:Calorimetry                           7.94e-05      0.0279509      2.31953     0.000697294    0.230669        100    
reco:pmtrackpid:Chi2ParticleID                         3.519e-05    0.000273092   0.00757844    0.000128029   0.000773226      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000213059     1.67076       54.5244      0.207545       5.83706        100    
reco:pmtrajfitcalo:Calorimetry                        7.6652e-05     0.0394884      2.93285     0.00104909     0.292615        100    
reco:pmtrajfitpid:Chi2ParticleID                      3.4419e-05    0.000164944   0.00166435    0.000123589   0.000185584      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00109445      3.82391       235.927      0.190723       23.6664        100    
reco:pmtracktccalo:Calorimetry                        7.9998e-05     0.0162211      1.18477     0.000676627    0.117912        100    
reco:pmtracktcpid:Chi2ParticleID                      3.4451e-05    0.000246163   0.00617125    0.000120749   0.000628087      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000177302     2.68843       172.183       0.27833       17.138         100    
reco:pmtrajfittccalo:Calorimetry                      7.5085e-05     0.0361768      2.91431     0.000822197    0.290054        100    
reco:pmtrajfittcpid:Chi2ParticleID                    3.4077e-05    0.000321385   0.00848885    0.000126782   0.000879582      100    
reco:blurredcluster:BlurredClustering                 7.1285e-05      1.00502       13.2988      0.376851       1.71124        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000122539     11.5182       363.821       2.44401       37.7448        100    
reco:emshower:EMShower                                0.000164104    0.0240914     0.716384     0.00322359     0.0761204       100    
reco:cvnmap:CVNMapper                                 2.1157e-05    0.00871896      0.06864     0.00966599    0.00845482       100    
reco:cvneva:CVNEvaluator                              1.9003e-05     0.506121      0.873686      0.698847       0.33283        100    
reco:energyrecnumu:EnergyReco                         0.000278014    0.0307563      1.00381     0.00339382     0.113729        100    
reco:energyrecnue:EnergyReco                          0.000160863   0.00301659      0.17087     0.000504749    0.0169837       100    
reco:energyrecnc:EnergyReco                           0.00215788    0.00516541     0.188323     0.00249262     0.0185277       100    
reco:energyrecnumurange:EnergyReco                    0.000148215    0.0292301      1.00553     0.00227379     0.113297        100    
reco:energyrecnumumcs:EnergyReco                      0.00168202     0.0303152      1.00302     0.00286773     0.112799        100    
reco:opdec:Deconvolution                               0.0661329     0.328386      0.620988      0.340971      0.105448        100    
reco:ophitspe:OpHitFinderDeco                          0.190249       1.32678       2.60631       1.44331      0.524934        100    
reco:opflash:OpFlashFinder                            0.000120848   0.00126114    0.00343703    0.00130918    0.000630056      100    
reco:rns:RandomNumberSaver                            1.7085e-05    2.48524e-05   0.000266879   1.9432e-05    2.48614e-05      100    
reco:anglereconue:NuAngularReco                       0.000125654    0.0029607     0.202123     0.000302809    0.0200926       100    
reco:anglereconumu:NuAngularReco                      8.3792e-05    0.00286175     0.203479     0.000212654    0.0202339       100    
reco:anglereconuepfps:NuAngularReco                   0.000126335   0.00297081      0.20309     0.000330201    0.020184        100    
reco:anglereconumupfps:NuAngularReco                  0.000119063   0.00291437     0.201023     0.00028209     0.0199841       100    
[art]:TriggerResults:TriggerResultInserter            1.0525e-05    1.27792e-05   6.4571e-05    1.15145e-05   5.9211e-06       100    
end_path:out1:RootOutput                               2.741e-06    4.21052e-06   1.6583e-05    3.2475e-06    1.95945e-06      100    
end_path:out1:RootOutput(write)                       0.00832719     0.221918       1.99273      0.227661      0.218742        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8803.05 MB
  Peak resident set size usage (VmHWM): 2704.57 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_1198_20231203T160707Z_gen_g4_detsim_hitreco__20240508T072521Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_1198_20231203T160707Z_gen_g4_detsim_hitreco__20240508T072521Z_reco2_reco_2024-11-23T_040053Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6414162_1198_20231203T160707Z_gen_g4_detsim_hitreco__20240508T072521Z_reco2_reco_data_2024-11-23T_040053Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work