Jobsub ID 280013.136@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	280013.136@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	3836
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-10-25 15:20:11
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce505
Last heartbeat	2024-10-25 17:16:46
From worker node	Hostname	b9p03p9423.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	aborted
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-10-25 16:14:56
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74511854_310_20231203T165058Z_gen_g4_detsim_hitreco__20240509T194949Z_reco2.root
Outputting started	2024-10-25 17:16:25
Output files
Finished	2024-10-25 17:16:46
List job events Wrapper job log
Jobscript log (last 10,000 characters)

CEST run: 74511854 subRun: 1 event: 31100
1st wire C:0 T:22 P:2 W:812 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  25-Oct-2024 19:15:44 CEST run: 74511854 subRun: 1 event: 31100
1st wire C:0 T:22 P:2 W:998 does not exist (max wire number: 480)
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  25-Oct-2024 19:15:45 CEST run: 74511854 subRun: 1 event: 31100
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  25-Oct-2024 19:15:45 CEST run: 74511854 subRun: 1 event: 31100
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  25-Oct-2024 19:15:45 CEST run: 74511854 subRun: 1 event: 31100
TuneFullTree failed.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  25-Oct-2024 19:15:45 CEST run: 74511854 subRun: 1 event: 31100
0 enabled hits in AverageDist2 calculation.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  25-Oct-2024 19:15:45 CEST run: 74511854 subRun: 1 event: 31100
Track empty.
%MSG
%MSG-e pma::Track3D:  PMAlgTrajFitter:pmtrajfittc@BeginModule  25-Oct-2024 19:15:45 CEST run: 74511854 subRun: 1 event: 31100
TuneFullTree failed.
%MSG
Boundary wire vector sizes: 2214, 1563, 2121
minwire 0: 1362
minwire 1: 673
minwire 2: 1104
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.00146279, 0.000423277, 0.998114, 
Output 1: 0.381757, 0.424657, 0.130416, 0.06317, 
Output 2: 0.78256, 0.21744, 

781
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 3086!
25-Oct-2024 19:16:24 CEST  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74511854_310_20231203T165058Z_gen_g4_detsim_hitreco__20240509T194949Z_reco2_reco_data_2024-10-25T_161500Z.root"
25-Oct-2024 19:16:24 CEST  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/ff/4a/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74511854_310_20231203T165058Z_gen_g4_detsim_hitreco__20240509T194949Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.602016       34.6394       891.151       8.10541       104.563        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0167773     0.0418493     0.104109      0.0424032     0.0139204       100    
reco:linecluster:LineCluster                           0.0952831     0.794598       42.0539      0.188913       4.37184        100    
reco:trajcluster:TrajCluster                           0.0475193      1.60089       72.1615      0.0612352      8.09285        100    
reco:pandora:StandardPandora                           0.0752038      3.11105       111.361       1.01127       11.7646        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000113622   0.00749285     0.126021     0.00150632     0.0193242       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000151603    0.0178514     0.407098     0.00312831     0.048682        100    
reco:pandoracalo:Calorimetry                          9.0601e-05     0.0383057      1.72006     0.000945624    0.191369        100    
reco:pandorapid:Chi2ParticleID                         3.345e-05      0.00016     0.00154237    0.000109912   0.00019579       100    
reco:trkshowersplit:TrackShowerHits                   8.9722e-05     0.0377005      1.34247     0.00076451     0.155494        100    
reco:pmtrack:PMAlgTrackMaker                          0.000938949     4.11253       156.784      0.171557       17.3225        100    
reco:pmtrackcalo:Calorimetry                          6.0711e-05     0.018968      0.619944     0.000717365    0.075345        100    
reco:pmtrackpid:Chi2ParticleID                        2.3181e-05    0.000275146   0.00308455    0.000105622   0.000486011      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000162153     2.71613       96.0908      0.271888       10.7603        100    
reco:pmtrajfitcalo:Calorimetry                        5.6721e-05     0.0410142      1.81826     0.00130385     0.201175        100    
reco:pmtrajfitpid:Chi2ParticleID                       2.345e-05    0.000163598   0.000848237   0.000113953   0.000161511      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000746615     4.69805       144.552      0.251658       18.2298        100    
reco:pmtracktccalo:Calorimetry                        5.7731e-05     0.0113196     0.267313     0.000820352    0.036037        100    
reco:pmtracktcpid:Chi2ParticleID                       2.243e-05    0.000250522   0.00314941    0.000122838   0.000423704      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000145343     2.17104       59.2415      0.263532       7.17499        100    
reco:pmtrajfittccalo:Calorimetry                      5.4581e-05     0.0258927     0.810557     0.00112452     0.103865        100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.238e-05    0.000341713   0.00421396    0.000135262   0.000652112      100    
reco:blurredcluster:BlurredClustering                 0.000130662    0.945871       6.51414      0.443099       1.21753        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000127672     11.5953       185.949       2.49628       26.9977        100    
reco:emshower:EMShower                                0.000177554    0.0209378     0.188618     0.00342257     0.0347523       100    
reco:cvnmap:CVNMapper                                 1.8891e-05     0.0102077     0.0411192     0.0136236    0.00818994       100    
reco:cvneva:CVNEvaluator                               1.449e-05     0.550668       1.01911        0.748       0.365229        100    
reco:energyrecnumu:EnergyReco                         0.000241595    0.0612461      1.26363     0.00356149     0.203052        100    
reco:energyrecnue:EnergyReco                          0.000113883   0.00260328     0.0532227    0.000446019   0.00722387       100    
reco:energyrecnc:EnergyReco                           0.00146552    0.00462959     0.0602233    0.00265214    0.00775445       100    
reco:energyrecnumurange:EnergyReco                    0.000130753    0.0592089      1.2487      0.00199735     0.200224        100    
reco:energyrecnumumcs:EnergyReco                      0.000549121    0.0588681      1.21012     0.00313361     0.194973        100    
reco:opdec:Deconvolution                               0.0520307     0.270809      0.970341      0.257166      0.118978        100    
reco:ophitspe:OpHitFinderDeco                          0.176285       1.39289       4.07662       1.36696      0.642208        100    
reco:opflash:OpFlashFinder                            0.000218864   0.00116796    0.00360788    0.00112532    0.000554718      100    
reco:rns:RandomNumberSaver                             1.562e-05    3.45011e-05   0.000268475   3.2806e-05    2.46923e-05      100    
reco:anglereconue:NuAngularReco                       0.000136273   0.00201841     0.0440126    0.000340882   0.00600583       100    
reco:anglereconumu:NuAngularReco                      6.2392e-05    0.00188565     0.044365     0.000213914   0.00618573       100    
reco:anglereconuepfps:NuAngularReco                   0.000104433    0.0020873     0.0444903    0.000313011   0.00668211       100    
reco:anglereconumupfps:NuAngularReco                  8.7331e-05    0.00189409     0.044286     0.00025012    0.00595389       100    
[art]:TriggerResults:TriggerResultInserter             8.18e-06     1.10041e-05   3.6901e-05    1.02405e-05    3.663e-06       100    
end_path:out1:RootOutput                               2.71e-06     5.81638e-06   1.6291e-05     5.835e-06    1.8006e-06       100    
end_path:out1:RootOutput(write)                       0.00959917      0.20956       1.30819      0.185879      0.198789        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21813.9 MB
  Peak resident set size usage (VmHWM): 2393.52 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74511854_310_20231203T165058Z_gen_g4_detsim_hitreco__20240509T194949Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74511854_310_20231203T165058Z_gen_g4_detsim_hitreco__20240509T194949Z_reco2_reco_2024-10-25T_161500Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74511854_310_20231203T165058Z_gen_g4_detsim_hitreco__20240509T194949Z_reco2_reco_data_2024-10-25T_161500Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
work