Jobsub ID 108551.75@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108551.75@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-22 23:16:35
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce507
Last heartbeat	2024-11-23 00:16:10
From worker node	Hostname	b9p30p0342.cern.ch
	cpuinfo	AMD EPYC 7302 16-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	stalled
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-22 23:34:20
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_819_20231202T085034Z_gen_g4_detsim_hitreco__20240507T205251Z_reco2.root
Outputting started	2024-11-23 00:16:10
Output files
Finished	2024-11-23 00:34:32
List job events Wrapper job log
Jobscript log (last 10,000 characters)

wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:16:00 CET run: 6400837 subRun: 1 event: 82000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:16:00 CET run: 6400837 subRun: 1 event: 82000
1st wire C:0 T:9 P:2 W:791 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:16:00 CET run: 6400837 subRun: 1 event: 82000
1st wire C:0 T:9 P:2 W:822 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:16:00 CET run: 6400837 subRun: 1 event: 82000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:16:00 CET run: 6400837 subRun: 1 event: 82000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:16:00 CET run: 6400837 subRun: 1 event: 82000
1st wire C:0 T:9 P:2 W:748 does not exist (max wire number: 480)
%MSG
Boundary wire vector sizes: 188, 160, 198
minwire 0: 1247
minwire 1: 1855
minwire 2: 1003
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0606877, 0.000625084, 0.938687, 
Output 1: 0.713951, 0.27483, 0.0109533, 0.000265248, 
Output 2: 0.929978, 0.0700221, 

594
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2148!
23-Nov-2024 01:16:09 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_819_20231202T085034Z_gen_g4_detsim_hitreco__20240507T205251Z_reco2_reco_data_2024-11-22T_233425Z.root"
23-Nov-2024 01:16:09 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/16/67/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_819_20231202T085034Z_gen_g4_detsim_hitreco__20240507T205251Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2.30894       23.0281       657.962       9.83717       70.189         100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0169662     0.0418711     0.161223      0.0410863     0.0181529       100    
reco:linecluster:LineCluster                           0.151351      0.357445       13.1852      0.173951       1.33074        100    
reco:trajcluster:TrajCluster                           0.047573      0.767991       41.2605      0.0587022      4.46585        100    
reco:pandora:StandardPandora                           0.915499       3.09689       151.164       1.41456       14.9248        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000131871   0.00415249     0.122314      0.0013818     0.0133665       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000218063   0.00794107     0.160567     0.00297759     0.0191157       100    
reco:pandoracalo:Calorimetry                          0.000104882    0.0138628     0.694153     0.00103403     0.0734502       100    
reco:pandorapid:Chi2ParticleID                        3.3721e-05    0.000162254   0.000954762   0.000140487   0.000138366      100    
reco:trkshowersplit:TrackShowerHits                   9.5311e-05     0.0180641      0.90836     0.000701004    0.0985751       100    
reco:pmtrack:PMAlgTrackMaker                           0.0011768      1.99852       95.2635      0.153917       9.93887        100    
reco:pmtrackcalo:Calorimetry                          6.0761e-05    0.00783536     0.337789     0.000851257    0.0381849       100    
reco:pmtrackpid:Chi2ParticleID                         2.397e-05    0.000247094   0.00343141    0.000142802   0.000445684      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000223943     1.69593       71.3344      0.209809       7.44657        100    
reco:pmtrajfitcalo:Calorimetry                        7.6941e-05     0.0156296     0.887109     0.00124533     0.091195        100    
reco:pmtrajfitpid:Chi2ParticleID                       2.936e-05    0.000168089   0.00109385    0.000139086   0.000134255      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00102215      2.08794       83.6194       0.21293       9.21257        100    
reco:pmtracktccalo:Calorimetry                        6.2961e-05    0.00411381     0.106571     0.000881121    0.0146671       100    
reco:pmtracktcpid:Chi2ParticleID                       2.603e-05    0.000211416    0.0019023    0.000127462   0.000280881      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000167062     1.17454       29.7843      0.202682       3.69543        100    
reco:pmtrajfittccalo:Calorimetry                       6.028e-05     0.0118034      0.55885     0.00102853     0.0605073       100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.779e-05    0.000291984   0.00442665    0.000142207   0.000551325      100    
reco:blurredcluster:BlurredClustering                 0.00318032      0.85657       6.37747      0.422821       1.22219        100    
reco:emtrkmichelid:EmTrackMichelId                     0.224397       7.92781       155.853       3.01742       19.3653        100    
reco:emshower:EMShower                                0.000308604    0.0252675     0.328995     0.00284796     0.047205        100    
reco:cvnmap:CVNMapper                                  2.001e-05     0.0118321      0.06971     0.00863712     0.011866        100    
reco:cvneva:CVNEvaluator                               1.605e-05     0.613773       1.18384      0.827244      0.419054        100    
reco:energyrecnumu:EnergyReco                         0.000291623    0.0303705      0.98921     0.00430539     0.125187        100    
reco:energyrecnue:EnergyReco                          0.000119591   0.00142386     0.039107     0.000504507   0.00407678       100    
reco:energyrecnc:EnergyReco                           0.00176727    0.00382831     0.044086     0.00297352    0.00440309       100    
reco:energyrecnumurange:EnergyReco                    0.000138711    0.0282753     0.990138      0.0029057     0.124149        100    
reco:energyrecnumumcs:EnergyReco                      0.00116321     0.0299086     0.988942     0.00403296     0.122833        100    
reco:opdec:Deconvolution                               0.0774285     0.307023       1.62992      0.294383      0.162447        100    
reco:ophitspe:OpHitFinderDeco                           0.24162       1.65802       3.54294       1.55265      0.731738        100    
reco:opflash:OpFlashFinder                            0.000316724   0.00142509    0.00822964    0.00132039    0.000907127      100    
reco:rns:RandomNumberSaver                             1.833e-05    4.79961e-05   0.000346394   4.1425e-05    4.02704e-05      100    
reco:anglereconue:NuAngularReco                       0.000166162   0.00106859     0.0327877    0.000362714   0.00342332       100    
reco:anglereconumu:NuAngularReco                      7.2131e-05    0.000854947    0.034084     0.000192123   0.00355458       100    
reco:anglereconuepfps:NuAngularReco                   0.000122512   0.00102646     0.0351026    0.00035934    0.00363171       100    
reco:anglereconumupfps:NuAngularReco                  0.000102021   0.000915235    0.0339973    0.000265829   0.00353591       100    
[art]:TriggerResults:TriggerResultInserter             8.83e-06     1.50533e-05   4.3051e-05     1.404e-05    5.83058e-06      100    
end_path:out1:RootOutput                               2.69e-06     7.45552e-06    1.855e-05     7.635e-06    2.81309e-06      100    
end_path:out1:RootOutput(write)                        0.0177303     0.220969      0.916094      0.236368       0.14983        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 12654.5 MB
  Peak resident set size usage (VmHWM): 2122.89 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_819_20231202T085034Z_gen_g4_detsim_hitreco__20240507T205251Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_819_20231202T085034Z_gen_g4_detsim_hitreco__20240507T205251Z_reco2_reco_2024-11-22T_233425Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6400837_819_20231202T085034Z_gen_g4_detsim_hitreco__20240507T205251Z_reco2_reco_data_2024-11-22T_233425Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work