Jobsub ID 108744.149@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108744.149@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:38:45
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce511
Last heartbeat	2024-11-23 05:42:25
From worker node	Hostname	b9p10p9366.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:45:03
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_887_20231204T231009Z_gen_g4_detsim_hitreco__20240510T032810Z_reco2.root
Jobscript	Exit code	0
	Real time	57m (3423s)
	CPU time	57m (3429s = 100%)
Outputting started	2024-11-23 05:42:07
Output files
Finished	2024-11-23 05:42:25
List job events Wrapper job log
Jobscript log (last 10,000 characters)

18431 subRun: 1 event: 88800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 06:41:49 CET run: 74518431 subRun: 1 event: 88800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 06:41:49 CET run: 74518431 subRun: 1 event: 88800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 06:41:49 CET run: 74518431 subRun: 1 event: 88800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 06:41:49 CET run: 74518431 subRun: 1 event: 88800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 06:41:49 CET run: 74518431 subRun: 1 event: 88800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 06:41:50 CET run: 74518431 subRun: 1 event: 88800
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 445, 592, 640
minwire 0: 48
minwire 1: 2182
minwire 2: 0
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0420654, 0.00833277, 0.949602, 
Output 1: 0.501006, 0.394868, 0.0883404, 0.0157861, 
Output 2: 0.0820199, 0.91798, 

382
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1085!
23-Nov-2024 06:42:05 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_887_20231204T231009Z_gen_g4_detsim_hitreco__20240510T032810Z_reco2_reco_data_2024-11-23T_044508Z.root"
23-Nov-2024 06:42:05 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/b8/15/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_887_20231204T231009Z_gen_g4_detsim_hitreco__20240510T032810Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2.79358       31.9695       351.651       10.0887       65.9726        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0172686     0.0432137     0.139781      0.0419322     0.0188492       100    
reco:linecluster:LineCluster                            0.15207      0.531561       9.02106       0.18112       1.45377        100    
reco:trajcluster:TrajCluster                           0.0484985      1.54718       37.5203      0.0653072      6.03497        100    
reco:pandora:StandardPandora                             1.006        2.41007       33.1748       1.29433       4.5671         100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000152592   0.00714625     0.0928604    0.00164781     0.0162946       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000204762    0.0126927      0.21326     0.00332509     0.026586        100    
reco:pandoracalo:Calorimetry                          0.000117491    0.0243117     0.571311     0.00134945     0.0804107       100    
reco:pandorapid:Chi2ParticleID                        4.2681e-05    0.000189376   0.00180183    0.000140092   0.00019475       100    
reco:trkshowersplit:TrackShowerHits                   0.000130832    0.0390836     0.815204     0.00092729     0.138498        100    
reco:pmtrack:PMAlgTrackMaker                          0.00121741      2.97532       48.4142      0.298674       8.6382         100    
reco:pmtrackcalo:Calorimetry                          7.1491e-05     0.0117625     0.239573     0.00115614     0.0367155       100    
reco:pmtrackpid:Chi2ParticleID                        2.6381e-05    0.000318056   0.00212187    0.000157642   0.000428562      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000191412     2.72641       41.7942      0.391859       7.40126        100    
reco:pmtrajfitcalo:Calorimetry                        5.9161e-05     0.0268253     0.677485     0.00153927     0.0890877       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.7941e-05    0.000209336   0.000751859   0.000161197   0.000155295      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00081812      3.61001       58.2138      0.253631       10.3807        100    
reco:pmtracktccalo:Calorimetry                        6.7741e-05    0.00765546     0.115971     0.00109993     0.0181259       100    
reco:pmtracktcpid:Chi2ParticleID                       2.559e-05    0.000303032   0.00194804    0.000158842   0.000369446      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.00175609      2.07653       26.1793      0.343308       4.90634        100    
reco:pmtrajfittccalo:Calorimetry                      0.000145372    0.017156      0.275995     0.00139784     0.0490667       100    
reco:pmtrajfittcpid:Chi2ParticleID                     5.317e-05    0.000418106   0.00296546    0.000188952   0.000579398      100    
reco:blurredcluster:BlurredClustering                  0.0100873      1.12914       6.50346      0.597433       1.33831        100    
reco:emtrkmichelid:EmTrackMichelId                     0.258762       11.2402       113.114       2.79473       21.5342        100    
reco:emshower:EMShower                                0.000304474    0.0345956     0.545158     0.00498153     0.0732163       100    
reco:cvnmap:CVNMapper                                  2.86e-05      0.0103646     0.0442059     0.0113647    0.00894546       100    
reco:cvneva:CVNEvaluator                              1.5161e-05     0.684571       1.21973      0.867537      0.427832        100    
reco:energyrecnumu:EnergyReco                         0.000302403    0.0829201      2.00015     0.00486026     0.282465        100    
reco:energyrecnue:EnergyReco                          0.000119471   0.00243783     0.0388816    0.00044264    0.00604314       100    
reco:energyrecnc:EnergyReco                           0.00157468    0.00499007     0.0410837    0.00313891    0.00623479       100    
reco:energyrecnumurange:EnergyReco                    0.000155552    0.0767875      1.95979     0.00270785     0.266661        100    
reco:energyrecnumumcs:EnergyReco                      0.00108112     0.0809323      1.96608     0.00409879     0.283616        100    
reco:opdec:Deconvolution                               0.132878       0.29312      0.633788      0.276818      0.0942348       100    
reco:ophitspe:OpHitFinderDeco                          0.625507       1.99275       4.00927       1.91143       0.83704        100    
reco:opflash:OpFlashFinder                            0.000388765   0.00165674     0.0047214    0.00161714    0.000716375      100    
reco:rns:RandomNumberSaver                             1.787e-05    4.75211e-05   0.000303133    4.653e-05    2.76942e-05      100    
reco:anglereconue:NuAngularReco                       0.000161562   0.00213696     0.0413674    0.000536296   0.00565169       100    
reco:anglereconumu:NuAngularReco                      6.4701e-05    0.00168541     0.0297061    0.000241098   0.00487655       100    
reco:anglereconuepfps:NuAngularReco                   0.000121062   0.00184277     0.0300673    0.000413445   0.00484505       100    
reco:anglereconumupfps:NuAngularReco                  8.7911e-05    0.00175028     0.0303968    0.000290553   0.00490824       100    
[art]:TriggerResults:TriggerResultInserter             8.51e-06     1.45907e-05    4.561e-05     1.366e-05    5.15639e-06      100    
end_path:out1:RootOutput                               2.96e-06     8.51273e-06   2.2231e-05     8.155e-06    3.09426e-06      100    
end_path:out1:RootOutput(write)                        0.0253145      0.25689       1.15413      0.237933      0.204588        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21651 MB
  Peak resident set size usage (VmHWM): 2208.37 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_887_20231204T231009Z_gen_g4_detsim_hitreco__20240510T032810Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_887_20231204T231009Z_gen_g4_detsim_hitreco__20240510T032810Z_reco2_reco_2024-11-23T_044508Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74518431_887_20231204T231009Z_gen_g4_detsim_hitreco__20240510T032810Z_reco2_reco_data_2024-11-23T_044508Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work