Jobsub ID 108738.63@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108738.63@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:22:32
Site	UK_RAL-PPD
Entry	CMSHTPC_T2_UK_SGrid_RALPP_hep208
Last heartbeat	2024-11-23 06:34:08
From worker node	Hostname	heplnc164.pp.rl.ac.uk
	cpuinfo	AMD EPYC 7763 64-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:33:07
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515285_419_20231201T123836Z_gen_g4_detsim_hitreco__20240507T190737Z_reco2.root
Jobscript	Exit code	0
	Real time	2h (7225s)
	CPU time	1h (5428s = 75%)
Outputting started	2024-11-23 06:33:33
Output files
Finished	2024-11-23 06:34:08
List job events Wrapper job log
Jobscript log (last 10,000 characters)

onCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:33:11 GMT run: 50515285 subRun: 1 event: 42000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:33:11 GMT run: 50515285 subRun: 1 event: 42000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:33:11 GMT run: 50515285 subRun: 1 event: 42000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:33:11 GMT run: 50515285 subRun: 1 event: 42000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:33:11 GMT run: 50515285 subRun: 1 event: 42000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:33:11 GMT run: 50515285 subRun: 1 event: 42000
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 06:33:11 GMT run: 50515285 subRun: 1 event: 42000
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 5838, 3098, 2775
minwire 0: 207
minwire 1: 1816
minwire 2: 46
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.0204431, 0.920501, 0.0590562, 
Output 1: 0.158929, 0.467781, 0.251199, 0.122092, 
Output 2: 0.0888708, 0.911129, 

772
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2807!
23-Nov-2024 06:33:30 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515285_419_20231201T123836Z_gen_g4_detsim_hitreco__20240507T190737Z_reco2_reco_data_2024-11-23T_043315Z.root"
23-Nov-2024 06:33:30 GMT  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/07/e2/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515285_419_20231201T123836Z_gen_g4_detsim_hitreco__20240507T190737Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              4.56982       60.4035       567.162       25.4485       102.109        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00580416     0.622247       3.12561      0.440594      0.632579        100    
reco:linecluster:LineCluster                           0.0486611      4.89255       46.1168       2.29799       6.85706        100    
reco:trajcluster:TrajCluster                           0.0131225      1.32053       26.9472      0.0288134      3.97984        100    
reco:pandora:StandardPandora                            1.4974        4.87337       28.2775       3.89994       3.89003        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000295465   0.00902118     0.103561     0.00193303     0.0167304       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000486063    0.0142067     0.143319     0.00358705     0.0251796       100    
reco:pandoracalo:Calorimetry                          0.000223961    0.0269543     0.536217     0.00120137     0.0862571       100    
reco:pandorapid:Chi2ParticleID                        6.9711e-05    0.000409619   0.00392956    0.000291342   0.000430335      100    
reco:trkshowersplit:TrackShowerHits                   0.000213701    0.0299931     0.744496     0.000880604    0.103715        100    
reco:pmtrack:PMAlgTrackMaker                          0.00176393      3.25038       76.0936      0.0876044      10.5267        100    
reco:pmtrackcalo:Calorimetry                          8.5341e-05     0.0125787     0.265128     0.00145072     0.0406522       100    
reco:pmtrackpid:Chi2ParticleID                        4.1067e-05    0.000580884   0.00508505    0.000330962   0.000921335      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000336713     2.84951       50.6441      0.136797       8.19486        100    
reco:pmtrajfitcalo:Calorimetry                        0.000123812    0.0331279     0.787576      0.0016628     0.112672        100    
reco:pmtrajfitpid:Chi2ParticleID                      5.4623e-05    0.000416471    0.0013558    0.000338987   0.000279542      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00142795      3.68695       61.0871      0.0811901      10.7485        100    
reco:pmtracktccalo:Calorimetry                        0.000108714   0.00805484     0.126632     0.00156004     0.0196847       100    
reco:pmtracktcpid:Chi2ParticleID                       4.786e-05    0.000560021   0.00571708    0.000339352   0.000791106      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000231335     1.77013       32.6596      0.187476       4.76954        100    
reco:pmtrajfittccalo:Calorimetry                      6.9771e-05     0.0165265     0.269009      0.001554      0.0467553       100    
reco:pmtrajfittcpid:Chi2ParticleID                    3.2191e-05    0.000763199   0.00959219    0.000352603   0.00130602       100    
reco:blurredcluster:BlurredClustering                 0.000459303     1.45317       9.66624      0.613985       2.11564        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000211377     26.4048       307.658       4.3163        55.7583        100    
reco:emshower:EMShower                                0.000300684    0.0330187     0.285211     0.00934304     0.0509872       100    
reco:cvnmap:CVNMapper                                 3.9574e-05     0.0254665     0.0730924      0.0276       0.0243771       100    
reco:cvneva:CVNEvaluator                              3.9264e-05      1.41856       3.89817       1.96756       1.28558        100    
reco:energyrecnumu:EnergyReco                         0.000833666    0.0378482      1.11523     0.00918057     0.125369        100    
reco:energyrecnue:EnergyReco                          0.000210685   0.00314571     0.0372213    0.000803099   0.00630257       100    
reco:energyrecnc:EnergyReco                           0.00289332    0.00651973     0.0366211     0.0046394    0.00613597       100    
reco:energyrecnumurange:EnergyReco                    0.000376517    0.0332305       1.153      0.00355303     0.128914        100    
reco:energyrecnumumcs:EnergyReco                      0.00273412     0.0352883      1.21591     0.00436398      0.13326        100    
reco:opdec:Deconvolution                               0.0489677      2.83207       14.3344       2.37089       2.56746        100    
reco:ophitspe:OpHitFinderDeco                          0.422893       4.36031       11.2734       4.29334       2.4777         100    
reco:opflash:OpFlashFinder                            0.000723189   0.00333998     0.0136357    0.00278688    0.00205655       100    
reco:rns:RandomNumberSaver                            4.1558e-05    0.000117956   0.000851179   9.92265e-05   8.55149e-05      100    
reco:anglereconue:NuAngularReco                       0.000340319   0.00253436     0.0445477    0.000913922   0.00523306       100    
reco:anglereconumu:NuAngularReco                      9.5409e-05    0.00172377     0.0376503    0.000489235    0.0043578       100    
reco:anglereconuepfps:NuAngularReco                   0.000260419   0.00202743     0.0367457    0.000812171   0.00426048       100    
reco:anglereconumupfps:NuAngularReco                  0.000198944   0.00188036     0.0403355    0.000576463   0.00458842       100    
[art]:TriggerResults:TriggerResultInserter            1.6832e-05    3.67332e-05   0.00012216    2.9591e-05    2.14046e-05      100    
end_path:out1:RootOutput                               6.492e-06    1.92353e-05   7.4359e-05    1.62655e-05   1.12106e-05      100    
end_path:out1:RootOutput(write)                        0.0223734     0.324494       1.15635      0.335851      0.261142        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 39362.1 MB
  Peak resident set size usage (VmHWM): 2241.68 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515285_419_20231201T123836Z_gen_g4_detsim_hitreco__20240507T190737Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515285_419_20231201T123836Z_gen_g4_detsim_hitreco__20240507T190737Z_reco2_reco_2024-11-23T_043315Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_50515285_419_20231201T123836Z_gen_g4_detsim_hitreco__20240507T190737Z_reco2_reco_data_2024-11-23T_043315Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work