Jobsub ID 108744.155@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108744.155@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:38:45
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce509
Last heartbeat	2024-11-23 05:26:25
From worker node	Hostname	b9p10p6463.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:43:22
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6470038_713_20231208T014558Z_gen_g4_detsim_hitreco__20240510T031603Z_reco2.root
Jobscript	Exit code	0
	Real time	42m (2568s)
	CPU time	47m (2853s = 111%)
Outputting started	2024-11-23 05:26:11
Output files
Finished	2024-11-23 05:26:25
List job events Wrapper job log
Jobscript log (last 10,000 characters)

:emshower@BeginModule  23-Nov-2024 06:26:06 CET run: 6470038 subRun: 1 event: 71400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:26:06 CET run: 6470038 subRun: 1 event: 71400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:26:06 CET run: 6470038 subRun: 1 event: 71400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:26:06 CET run: 6470038 subRun: 1 event: 71400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:26:06 CET run: 6470038 subRun: 1 event: 71400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:26:06 CET run: 6470038 subRun: 1 event: 71400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:26:06 CET run: 6470038 subRun: 1 event: 71400
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 5194, 5734, 3918
minwire 0: 604
minwire 1: 1175
minwire 2: 276
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0566401, 0.789555, 0.153805, 
Output 1: 0.315978, 0.532379, 0.127894, 0.0237484, 
Output 2: 0.0571227, 0.942877, 

894
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3462!
23-Nov-2024 06:26:10 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6470038_713_20231208T014558Z_gen_g4_detsim_hitreco__20240510T031603Z_reco2_reco_data_2024-11-23T_044328Z.root"
23-Nov-2024 06:26:10 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/bf/1e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6470038_713_20231208T014558Z_gen_g4_detsim_hitreco__20240510T031603Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.685678       23.6196       377.859       8.66012       48.0105        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0171755     0.0433529     0.0939255     0.043283      0.0150296       100    
reco:linecluster:LineCluster                           0.0953823     0.358999       11.5509       0.17894       1.1567         100    
reco:trajcluster:TrajCluster                           0.0477364     0.858295       45.2955      0.0641638      4.57155        100    
reco:pandora:StandardPandora                           0.0561636      1.75473       25.9575       1.0243        3.02022        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000119151   0.00503304     0.0662777    0.00134695     0.0104933       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000151491   0.00798466     0.0882398    0.00244364     0.0129543       100    
reco:pandoracalo:Calorimetry                           9.535e-05     0.0163688     0.633945     0.000975946    0.0685654       100    
reco:pandorapid:Chi2ParticleID                         3.294e-05    0.000138272   0.00162248    0.000101351   0.000168549      100    
reco:trkshowersplit:TrackShowerHits                    7.162e-05     0.0198668     0.705227     0.000759974    0.0764959       100    
reco:pmtrack:PMAlgTrackMaker                          0.000949236     2.30511       68.6674      0.193135       7.78442        100    
reco:pmtrackcalo:Calorimetry                           5.639e-05    0.00895917     0.161199      0.0007536     0.0277503       100    
reco:pmtrackpid:Chi2ParticleID                         2.202e-05    0.000242677   0.00180441    0.000111021    0.000351        100    
reco:pmtrajfit:PMAlgTrajFitter                        0.00015522      1.74765       35.1485      0.266867       4.47303        100    
reco:pmtrajfitcalo:Calorimetry                        5.8181e-05     0.0168169     0.548804     0.00152626     0.0620011       100    
reco:pmtrajfitpid:Chi2ParticleID                       2.275e-05    0.000163369   0.000786465   0.000129641   0.000133325      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000759125     2.55822       63.772        0.24134       7.83078        100    
reco:pmtracktccalo:Calorimetry                        5.7651e-05    0.00619429     0.0959417     0.0008122     0.0174875       100    
reco:pmtracktcpid:Chi2ParticleID                       2.488e-05    0.000224627    0.0019391    0.000129836   0.000318472      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000124521     1.37032       21.8969      0.255928       3.20666        100    
reco:pmtrajfittccalo:Calorimetry                       5.264e-05     0.0135383     0.257796     0.00100157     0.040719        100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.176e-05    0.000313655   0.00229195    0.000136021   0.000474248      100    
reco:blurredcluster:BlurredClustering                 0.000125501    0.927564       5.51005      0.409183       1.1675         100    
reco:emtrkmichelid:EmTrackMichelId                    9.9471e-05      8.89369       94.3209       2.75892       16.2879        100    
reco:emshower:EMShower                                0.00013518     0.0240601     0.230228     0.00513333     0.0402618       100    
reco:cvnmap:CVNMapper                                  2.056e-05     0.0087705     0.0316393     0.0057954    0.00786757       100    
reco:cvneva:CVNEvaluator                               1.416e-05     0.587229      0.988677      0.722673      0.337303        100    
reco:energyrecnumu:EnergyReco                         0.000270122    0.0426033      1.69121     0.00341572     0.178605        100    
reco:energyrecnue:EnergyReco                          0.000112891   0.00160285     0.0210216    0.000451493   0.00324971       100    
reco:energyrecnc:EnergyReco                           0.00156124     0.0034785     0.0252868    0.00243699    0.00358854       100    
reco:energyrecnumurange:EnergyReco                    0.000124781    0.0405674      1.70402      0.002095      0.179505        100    
reco:energyrecnumumcs:EnergyReco                      0.000584834    0.0421681      1.72207     0.00334274     0.181111        100    
reco:opdec:Deconvolution                                0.05522      0.262491      0.469408      0.266037      0.0698263       100    
reco:ophitspe:OpHitFinderDeco                          0.189008       1.49005       3.0352        1.48267      0.542169        100    
reco:opflash:OpFlashFinder                            0.000173601   0.00116822    0.00589958    0.00111479    0.000631608      100    
reco:rns:RandomNumberSaver                             1.559e-05    3.1739e-05    0.000410923    2.852e-05     3.913e-05       100    
reco:anglereconue:NuAngularReco                       0.000138231   0.00121319     0.0164698    0.000389443   0.00253649       100    
reco:anglereconumu:NuAngularReco                      5.7651e-05    0.00103728     0.0162261    0.000205136   0.00246113       100    
reco:anglereconuepfps:NuAngularReco                    9.622e-05    0.00116599     0.0164275    0.000336672    0.0024748       100    
reco:anglereconumupfps:NuAngularReco                   8.28e-05     0.00107993     0.016268     0.000241887   0.00244754       100    
[art]:TriggerResults:TriggerResultInserter             8.16e-06     1.17697e-05    5.432e-05     1.055e-05    5.34472e-06      100    
end_path:out1:RootOutput                               2.47e-06     5.38432e-06   2.7271e-05     5.56e-06     2.8482e-06       100    
end_path:out1:RootOutput(write)                        0.0108709     0.195466      0.774539      0.185784       0.12876        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21737.3 MB
  Peak resident set size usage (VmHWM): 2312.74 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6470038_713_20231208T014558Z_gen_g4_detsim_hitreco__20240510T031603Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6470038_713_20231208T014558Z_gen_g4_detsim_hitreco__20240510T031603Z_reco2_reco_2024-11-23T_044328Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6470038_713_20231208T014558Z_gen_g4_detsim_hitreco__20240510T031603Z_reco2_reco_data_2024-11-23T_044328Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work