Jobsub ID 108511.16@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108511.16@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-22 22:09:40
Site	UK_Manchester
Entry	UBoone_T2_UK_Manchester_ce02
Last heartbeat	2024-11-22 23:30:33
From worker node	Hostname	wn2208240.tier2.hep.manchester.ac.uk
	cpuinfo	AMD EPYC 7513 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	Inner Apptainer?	True
Job state	aborted
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-22 22:29:54
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472771_622_20231208T090704Z_gen_g4_detsim_hitreco__20240510T062701Z_reco2.root
Outputting started	2024-11-22 23:23:55
Output files
Finished	2024-11-22 23:30:33
List job events Wrapper job log
Jobscript log (last 10,000 characters)

reIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:23:44 GMT run: 6472771 subRun: 1 event: 62300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:23:44 GMT run: 6472771 subRun: 1 event: 62300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:23:44 GMT run: 6472771 subRun: 1 event: 62300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:23:44 GMT run: 6472771 subRun: 1 event: 62300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:23:44 GMT run: 6472771 subRun: 1 event: 62300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:23:44 GMT run: 6472771 subRun: 1 event: 62300
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:23:44 GMT run: 6472771 subRun: 1 event: 62300
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 412, 419, 382
minwire 0: 1797
minwire 1: 410
minwire 2: 1653
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.320768, 0.619354, 0.0598779, 
Output 1: 0.138558, 0.707075, 0.146572, 0.00779509, 
Output 2: 0.199661, 0.80034, 

601
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2203!
22-Nov-2024 23:23:53 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472771_622_20231208T090704Z_gen_g4_detsim_hitreco__20240510T062701Z_reco2_reco_data_2024-11-22T_222959Z.root"
22-Nov-2024 23:23:53 GMT  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/3f/a4/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472771_622_20231208T090704Z_gen_g4_detsim_hitreco__20240510T062701Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.44746       29.2992       235.129       10.3651       48.3242        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0106057     0.0995622     0.783005      0.0370561     0.137383        100    
reco:linecluster:LineCluster                           0.0682813     0.491103       4.92662      0.146723      0.816038        100    
reco:trajcluster:TrajCluster                           0.0299547     0.626797       13.4526      0.0416958      2.08014        100    
reco:pandora:StandardPandora                           0.0836245      2.18013       16.0422       1.46927       2.47331        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.00015958    0.00661255     0.0896245    0.00154677     0.0140834       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000193172    0.0195481     0.388895     0.00325106     0.0522569       100    
reco:pandoracalo:Calorimetry                          0.000127249    0.0178643     0.301375     0.00114453     0.0506657       100    
reco:pandorapid:Chi2ParticleID                        4.2941e-05    0.000211148   0.00203595    0.000152866   0.000232208      100    
reco:trkshowersplit:TrackShowerHits                   9.2544e-05     0.0225317     0.373935     0.000729995    0.0712788       100    
reco:pmtrack:PMAlgTrackMaker                          0.00108576      2.26331       26.818       0.262564       5.36905        100    
reco:pmtrackcalo:Calorimetry                          7.3177e-05    0.00934158     0.159048      0.0011087     0.0251248       100    
reco:pmtrackpid:Chi2ParticleID                        2.8313e-05    0.00031148    0.00241507    0.000171718   0.000395633      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000195196     1.95861       23.8397      0.266232       4.3079         100    
reco:pmtrajfitcalo:Calorimetry                         6.392e-05     0.0181837     0.309295     0.00138393     0.0492999       100    
reco:pmtrajfitpid:Chi2ParticleID                       3.179e-05    0.000229092   0.00110691    0.000174116   0.000195075      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000885222     3.24442       47.3674      0.273716       8.21758        100    
reco:pmtracktccalo:Calorimetry                        7.9479e-05    0.00847522     0.134701      0.0011001     0.0226889       100    
reco:pmtracktcpid:Chi2ParticleID                      2.7953e-05    0.000315878   0.00262747    0.000172343   0.000419864      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000168176     1.6628        16.283       0.270686       3.49895        100    
reco:pmtrajfittccalo:Calorimetry                      6.0704e-05     0.0136288     0.193256     0.00107221     0.0374449       100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.4626e-05    0.000401005   0.00312766    0.000182993   0.000586041      100    
reco:blurredcluster:BlurredClustering                 0.000137899     1.46398       8.72989      0.647061       1.93921        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000109836     11.0708       91.3112       2.83058       19.9506        100    
reco:emshower:EMShower                                0.000171031    0.0398112     0.441035     0.00601318     0.0720823       100    
reco:cvnmap:CVNMapper                                 2.0368e-05     0.0137037     0.0571786     0.0128337     0.012247        100    
reco:cvneva:CVNEvaluator                               1.61e-05      0.840848       1.3336        1.0746       0.467181        100    
reco:energyrecnumu:EnergyReco                         0.000392677    0.0414257      1.46894     0.00479218     0.164042        100    
reco:energyrecnue:EnergyReco                          0.000155512   0.00227714     0.033113     0.000619338   0.00451108       100    
reco:energyrecnc:EnergyReco                           0.00163516    0.00503228     0.0261379    0.00359444    0.00422434       100    
reco:energyrecnumurange:EnergyReco                    0.000220083    0.0385651      1.46001     0.00316554     0.161497        100    
reco:energyrecnumumcs:EnergyReco                      0.00183064     0.0407539      1.47221     0.00469553     0.163194        100    
reco:opdec:Deconvolution                               0.0522331     0.446091       3.60814      0.264159      0.540641        100    
reco:ophitspe:OpHitFinderDeco                          0.164229       2.31798       5.19465       2.12328       1.07223        100    
reco:opflash:OpFlashFinder                            0.000132749   0.00164716    0.00599272    0.00151063    0.000796533      100    
reco:rns:RandomNumberSaver                            2.2953e-05    5.26712e-05   0.000481424   4.6247e-05    4.59194e-05      100    
reco:anglereconue:NuAngularReco                       0.00017576     0.0019315     0.0475171    0.000450024   0.00535532       100    
reco:anglereconumu:NuAngularReco                      9.5639e-05     0.0014688     0.0301359    0.000235602   0.00386129       100    
reco:anglereconuepfps:NuAngularReco                   0.000157425   0.00163419     0.0296603    0.00039723    0.00381267       100    
reco:anglereconumupfps:NuAngularReco                  0.000124143   0.00153911     0.0297699    0.00030378     0.0038307       100    
[art]:TriggerResults:TriggerResultInserter             8.927e-06    1.54279e-05   9.4958e-05    1.4462e-05    8.68593e-06      100    
end_path:out1:RootOutput                               2.956e-06    6.78058e-06   2.9535e-05     6.452e-06    3.30356e-06      100    
end_path:out1:RootOutput(write)                        0.0182056     0.322921       1.07676      0.313138      0.213351        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21627.2 MB
  Peak resident set size usage (VmHWM): 2167.16 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472771_622_20231208T090704Z_gen_g4_detsim_hitreco__20240510T062701Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472771_622_20231208T090704Z_gen_g4_detsim_hitreco__20240510T062701Z_reco2_reco_2024-11-22T_222959Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6472771_622_20231208T090704Z_gen_g4_detsim_hitreco__20240510T062701Z_reco2_reco_data_2024-11-22T_222959Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work