Jobsub ID 301665.99@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	301665.99@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 03:06:47
Site	UK_Brunel
Entry	CMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat	2024-11-23 03:46:59
From worker node	Hostname	wn-b2-16-01.brunel.ac.uk
	cpuinfo	Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 03:09:43
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_503_20231203T175534Z_gen_g4_detsim_hitreco__20240509T215638Z_reco2.root
Jobscript	Exit code	0
	Real time	37m (2221s)
	CPU time	32m (1924s = 86%)
Outputting started	2024-11-23 03:46:45
Output files
Finished	2024-11-23 03:46:59
List job events Wrapper job log
Jobscript log (last 10,000 characters)

Run: 1 event: 50400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 03:46:37 GMT run: 6419482 subRun: 1 event: 50400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 03:46:38 GMT run: 6419482 subRun: 1 event: 50400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 03:46:38 GMT run: 6419482 subRun: 1 event: 50400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 03:46:38 GMT run: 6419482 subRun: 1 event: 50400
1st wire C:0 T:9 P:2 W:983 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 03:46:38 GMT run: 6419482 subRun: 1 event: 50400
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 03:46:38 GMT run: 6419482 subRun: 1 event: 50400
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 64, 67, 73
minwire 0: 1445
minwire 1: 2083
minwire 2: 981
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.959177, 0.0153692, 0.0254542, 
Output 1: 0.0606652, 0.858367, 0.0734738, 0.00749387, 
Output 2: 0.986936, 0.0130638, 

547
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1906!
23-Nov-2024 03:46:43 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_503_20231203T175534Z_gen_g4_detsim_hitreco__20240509T215638Z_reco2_reco_data_2024-11-23T_030946Z.root"
23-Nov-2024 03:46:43 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/bb/47/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_503_20231203T175534Z_gen_g4_detsim_hitreco__20240509T215638Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.697424       18.2958       327.641       8.51292       35.9714        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0218289     0.0630673      0.76641      0.0523683     0.0844477       100    
reco:linecluster:LineCluster                           0.126496      0.271234       1.81526      0.231693      0.171699        100    
reco:trajcluster:TrajCluster                           0.0634694     0.463063       22.1616      0.0750907      2.24627        100    
reco:pandora:StandardPandora                           0.071546       1.55947       16.527        1.26686       1.60345        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000120733   0.00418219     0.118011     0.00118183     0.0124267       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000176784    0.0133849     0.550032     0.00232648     0.056054        100    
reco:pandoracalo:Calorimetry                          8.9972e-05    0.00774749     0.270896     0.000829269    0.0291057       100    
reco:pandorapid:Chi2ParticleID                        3.8903e-05    0.000134093   0.00123113    0.000108019   0.000136745      100    
reco:trkshowersplit:TrackShowerHits                   5.3924e-05     0.0074644     0.278803     0.000605057    0.0294217       100    
reco:pmtrack:PMAlgTrackMaker                          0.00114397      1.25155       44.8681      0.206404       4.61153        100    
reco:pmtrackcalo:Calorimetry                          8.0533e-05    0.00532323     0.161021     0.000704292    0.0175388       100    
reco:pmtrackpid:Chi2ParticleID                        3.2373e-05    0.000204493   0.00204893    0.000118532   0.000266914      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000174926     1.16627       22.4869       0.22743       3.14389        100    
reco:pmtrajfitcalo:Calorimetry                        7.4405e-05    0.00846663     0.290137      0.0011261     0.0311442       100    
reco:pmtrajfitpid:Chi2ParticleID                      3.2462e-05    0.000145869   0.000802623   0.000122585   0.000117116      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00103408      1.54217       57.3973      0.164906       5.95035        100    
reco:pmtracktccalo:Calorimetry                        7.8673e-05     0.0054146      0.20322     0.000761463    0.0210218       100    
reco:pmtracktcpid:Chi2ParticleID                      3.2494e-05    0.000194601   0.00164338    0.000128201   0.000226247      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000165418     0.96161       25.8956      0.279273       2.74266        100    
reco:pmtrajfittccalo:Calorimetry                      7.2121e-05      0.00827      0.328916     0.000865141    0.0340515       100    
reco:pmtrajfittcpid:Chi2ParticleID                    3.1932e-05    0.000239373   0.00223432    0.000123147   0.000315393      100    
reco:blurredcluster:BlurredClustering                 5.9273e-05     0.828733       6.86774       0.40362       1.15203        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000103221     7.84763       127.095       2.65683       15.2736        100    
reco:emshower:EMShower                                0.000177988    0.0212739     0.197539     0.00241353     0.0383339       100    
reco:cvnmap:CVNMapper                                 2.2233e-05    0.00955673     0.0460007    0.00949151    0.00786314       100    
reco:cvneva:CVNEvaluator                               1.833e-05     0.494422      0.929495      0.661509      0.310244        100    
reco:energyrecnumu:EnergyReco                         0.000318323    0.0154479     0.128708     0.00373894     0.0284938       100    
reco:energyrecnue:EnergyReco                          0.000152436   0.00158724     0.0381023    0.000560964   0.00401042       100    
reco:energyrecnc:EnergyReco                           0.00199725    0.00339901     0.0413943    0.00237743    0.00418644       100    
reco:energyrecnumurange:EnergyReco                    0.000198248    0.0141009     0.125342     0.00233112     0.0281757       100    
reco:energyrecnumumcs:EnergyReco                      0.00203941     0.0152261     0.125693     0.00308334     0.0283687       100    
reco:opdec:Deconvolution                               0.0683353     0.299369      0.524603      0.309387      0.104235        100    
reco:ophitspe:OpHitFinderDeco                          0.189834       1.20397       2.15189       1.19213      0.508604        100    
reco:opflash:OpFlashFinder                            0.000126135   0.00112701    0.00270354    0.00109848    0.000558168      100    
reco:rns:RandomNumberSaver                            1.6192e-05    2.60492e-05   0.000248276   2.1405e-05    2.35086e-05      100    
reco:anglereconue:NuAngularReco                       9.6889e-05    0.00103556     0.0330134    0.000268518   0.00344881       100    
reco:anglereconumu:NuAngularReco                      7.8831e-05    0.000943164    0.0327377    0.000184727   0.00341962       100    
reco:anglereconuepfps:NuAngularReco                   0.000114264   0.00105196     0.0329149    0.000279999   0.00343417       100    
reco:anglereconumupfps:NuAngularReco                  0.000112239    0.0010123     0.0329823    0.000259738   0.00344311       100    
[art]:TriggerResults:TriggerResultInserter             9.864e-06    1.26049e-05   3.9818e-05    1.18105e-05   3.5831e-06       100    
end_path:out1:RootOutput                               2.667e-06    3.94258e-06   1.4013e-05     3.156e-06    1.73219e-06      100    
end_path:out1:RootOutput(write)                       0.00807374     0.194807      0.798136      0.212297      0.124761        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8345.71 MB
  Peak resident set size usage (VmHWM): 2244.78 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_503_20231203T175534Z_gen_g4_detsim_hitreco__20240509T215638Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_503_20231203T175534Z_gen_g4_detsim_hitreco__20240509T215638Z_reco2_reco_2024-11-23T_030946Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6419482_503_20231203T175534Z_gen_g4_detsim_hitreco__20240509T215638Z_reco2_reco_data_2024-11-23T_030946Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work