Jobsub ID 108550.125@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108550.125@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-22 23:15:23
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce515
Last heartbeat	2024-11-23 00:18:42
From worker node	Hostname	b9p08p5554.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	stalled
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-22 23:33:35
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494003_743_20231201T170351Z_gen_g4_detsim_hitreco__20240507T201523Z_reco2.root
Outputting started	2024-11-23 00:18:42
Output files
Finished	2024-11-23 00:34:32
List job events Wrapper job log
Jobscript log (last 10,000 characters)

paring two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:18:29 CET run: 74494003 subRun: 1 event: 74400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:18:29 CET run: 74494003 subRun: 1 event: 74400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:18:29 CET run: 74494003 subRun: 1 event: 74400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:18:29 CET run: 74494003 subRun: 1 event: 74400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:18:29 CET run: 74494003 subRun: 1 event: 74400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 01:18:29 CET run: 74494003 subRun: 1 event: 74400
1st wire C:0 T:10 P:2 W:1130 does not exist (max wire number: 480)
%MSG
Boundary wire vector sizes: 628, 196, 417
minwire 0: 1027
minwire 1: 2301
minwire 2: 636
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.00596104, 0.000100368, 0.993939, 
Output 1: 0.0102879, 0.0972409, 0.535963, 0.356509, 
Output 2: 0.963912, 0.0360877, 

689
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2686!
23-Nov-2024 01:18:41 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494003_743_20231201T170351Z_gen_g4_detsim_hitreco__20240507T201523Z_reco2_reco_data_2024-11-22T_233341Z.root"
23-Nov-2024 01:18:41 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/61/40/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494003_743_20231201T170351Z_gen_g4_detsim_hitreco__20240507T201523Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.530019       25.0615       765.269       7.98636       84.0266        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0170905     0.101812       4.21361      0.0419481     0.450624        100    
reco:linecluster:LineCluster                           0.0946591     0.861997       61.7828       0.17521       6.14304        100    
reco:trajcluster:TrajCluster                           0.0471019      1.0127        66.5795      0.0553334      6.90215        100    
reco:pandora:StandardPandora                           0.0562388      2.20041       76.9737       1.17582       7.7926         100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000106931   0.00479972     0.127556     0.00105757     0.0150667       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000142242    0.0127256     0.182649     0.00216986     0.0293572       100    
reco:pandoracalo:Calorimetry                          9.4951e-05     0.023037       1.26445     0.000702065     0.13393        100    
reco:pandorapid:Chi2ParticleID                        3.4441e-05    0.000150159   0.000951442   0.000120067   0.000128811      100    
reco:trkshowersplit:TrackShowerHits                   6.9501e-05     0.0223254      1.26011     0.000508367    0.135929        100    
reco:pmtrack:PMAlgTrackMaker                           0.0010103      3.12095       168.817      0.119689       17.4525        100    
reco:pmtrackcalo:Calorimetry                          6.0471e-05    0.00792153     0.234962     0.000643048    0.0321685       100    
reco:pmtrackpid:Chi2ParticleID                         2.336e-05    0.000238932   0.00332012    0.000133207   0.000465841      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000159082     2.03413       73.7633      0.202037       8.40783        100    
reco:pmtrajfitcalo:Calorimetry                        5.8631e-05     0.0200558      1.00919     0.00102673     0.109484        100    
reco:pmtrajfitpid:Chi2ParticleID                       2.465e-05    0.000167576   0.000938712   0.000146372   0.000131519      100    
reco:pmtracktc:PMAlgTrackMaker                         0.0007955      2.79846       110.329      0.227828       13.3113        100    
reco:pmtracktccalo:Calorimetry                        5.8711e-05    0.00534647     0.132799     0.000709119    0.0164902       100    
reco:pmtracktcpid:Chi2ParticleID                      2.3931e-05    0.000218468   0.00220059    0.000126051   0.000322915      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000130692     1.39224       40.1732      0.205193       4.83024        100    
reco:pmtrajfittccalo:Calorimetry                       5.431e-05     0.0132042     0.626652     0.000820936    0.0701223       100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.1881e-05    0.00026777    0.00466912    0.000143477   0.000564688      100    
reco:blurredcluster:BlurredClustering                 9.2751e-05     0.744734       5.4834       0.419095       1.02519        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000106071     7.68535       147.661       1.7768        19.3028        100    
reco:emshower:EMShower                                0.000130942    0.0214537     0.466914     0.00283072     0.0548936       100    
reco:cvnmap:CVNMapper                                  2.128e-05    0.00916086     0.0369478     0.0109785    0.00684068       100    
reco:cvneva:CVNEvaluator                               1.587e-05     0.616884       1.12387      0.792389      0.394005        100    
reco:energyrecnumu:EnergyReco                         0.000341874    0.0687261      3.13114     0.00322778     0.354659        100    
reco:energyrecnue:EnergyReco                          0.000123791   0.00165685     0.039192     0.000421905   0.00481948       100    
reco:energyrecnc:EnergyReco                            0.0014556    0.00374463     0.0453086    0.00285519    0.00530288       100    
reco:energyrecnumurange:EnergyReco                    0.000153242    0.0670436      3.12493     0.00173307     0.354899        100    
reco:energyrecnumumcs:EnergyReco                      0.000937412    0.0660684      3.02365     0.00315109     0.339654        100    
reco:opdec:Deconvolution                               0.0879502     0.272529      0.755072      0.269008      0.088242        100    
reco:ophitspe:OpHitFinderDeco                          0.140697       1.66367       3.05205       1.60101      0.635532        100    
reco:opflash:OpFlashFinder                            0.000126462   0.00125964    0.00450158    0.00120405    0.000576276      100    
reco:rns:RandomNumberSaver                            1.7071e-05    3.59402e-05   0.000320014    3.268e-05    2.9187e-05       100    
reco:anglereconue:NuAngularReco                       0.000128321   0.00116622     0.0326274    0.00037803    0.00362849       100    
reco:anglereconumu:NuAngularReco                       6.082e-05    0.000917287    0.031462     0.000182427   0.00348031       100    
reco:anglereconuepfps:NuAngularReco                   0.000105231   0.00104427     0.0303359    0.000324824    0.0033552       100    
reco:anglereconumupfps:NuAngularReco                  9.6691e-05    0.000936463    0.0272158    0.000236948   0.00318993       100    
[art]:TriggerResults:TriggerResultInserter              8.1e-06     1.19558e-05    4.471e-05    1.10555e-05   4.22865e-06      100    
end_path:out1:RootOutput                               2.37e-06     7.70362e-06   2.3131e-05    7.1905e-06    2.92336e-06      100    
end_path:out1:RootOutput(write)                        0.0107327     0.200089      0.964778      0.209829      0.141138        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21737.4 MB
  Peak resident set size usage (VmHWM): 2320.81 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494003_743_20231201T170351Z_gen_g4_detsim_hitreco__20240507T201523Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494003_743_20231201T170351Z_gen_g4_detsim_hitreco__20240507T201523Z_reco2_reco_2024-11-22T_233341Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74494003_743_20231201T170351Z_gen_g4_detsim_hitreco__20240507T201523Z_reco2_reco_data_2024-11-22T_233341Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work