Jobsub ID 108512.180@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108512.180@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-22 22:14:30
Site	UK_Brunel
Entry	CMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat	2024-11-22 23:24:24
From worker node	Hostname	wn-a3-37-01.brunel.ac.uk
	cpuinfo	Intel(R) Xeon(R) Gold 6126 CPU @ 2.60GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	stalled
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-22 22:34:23
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_597_20231207T160917Z_gen_g4_detsim_hitreco__20240510T031533Z_reco2.root
Outputting started	2024-11-22 23:22:08
Output files
Finished	2024-11-22 23:33:58
List job events Wrapper job log
Jobscript log (last 10,000 characters)

in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:22:01 GMT run: 74568471 subRun: 1 event: 59800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:22:01 GMT run: 74568471 subRun: 1 event: 59800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  22-Nov-2024 23:22:01 GMT run: 74568471 subRun: 1 event: 59800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:22:04 GMT run: 74568471 subRun: 1 event: 59800
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  22-Nov-2024 23:22:04 GMT run: 74568471 subRun: 1 event: 59800
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 70, 87, 74
minwire 0: 386
minwire 1: 2038
minwire 2: 232
Used alternate method to get min and max wires due to vertex determination failure: 250, 449
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 237, 436
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max wires due to vertex determination failure: 114, 313
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.97788, 0.00446225, 0.0176581, 
Output 1: 0.856486, 0.125845, 0.0151645, 0.00250482, 
Output 2: 0.984625, 0.0153751, 

424
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1271!
22-Nov-2024 23:22:07 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_597_20231207T160917Z_gen_g4_detsim_hitreco__20240510T031533Z_reco2_reco_data_2024-11-22T_223431Z.root"
22-Nov-2024 23:22:07 GMT  Closed input file "root://xrootd1.esc.qmul.ac.uk:1094//dune/RSE/fardet-hd/18/b3/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_597_20231207T160917Z_gen_g4_detsim_hitreco__20240510T031533Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              1.88964       26.3592       448.611       8.63832       64.7556        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                0.00424345     0.0752317      0.67907      0.0246149     0.106313        100    
reco:linecluster:LineCluster                           0.0390295     0.608121       18.8087      0.0583581      2.21313        100    
reco:trajcluster:TrajCluster                           0.0108498      1.14964       34.235       0.0215725      5.3295         100    
reco:pandora:StandardPandora                           0.929018       2.17288       39.2269       1.07084       4.85907        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000119218   0.00557845     0.115444     0.00139712     0.0169213       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000154842   0.00848917     0.215653     0.00224506     0.0234675       100    
reco:pandoracalo:Calorimetry                          9.2697e-05     0.0218056     0.776776     0.000908713    0.0945865       100    
reco:pandorapid:Chi2ParticleID                        3.4142e-05    0.000120527   0.000990105   9.00215e-05   0.000120713      100    
reco:trkshowersplit:TrackShowerHits                   8.9447e-05     0.025384       0.67805     0.000586746    0.105762        100    
reco:pmtrack:PMAlgTrackMaker                          0.00105835      2.16696       72.9378      0.123377       8.57383        100    
reco:pmtrackcalo:Calorimetry                          6.8556e-05     0.0093462     0.250706     0.000705238    0.0350967       100    
reco:pmtrackpid:Chi2ParticleID                        2.7532e-05    0.000216901   0.00246354    9.6177e-05    0.000392322      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000171459     1.86574       48.232       0.161446       6.3734         100    
reco:pmtrajfitcalo:Calorimetry                        5.2914e-05     0.0220723      0.7545      0.00121427     0.0932762       100    
reco:pmtrajfitpid:Chi2ParticleID                      2.4546e-05    0.000133058   0.000654958   0.000110262   0.000110119      100    
reco:pmtracktc:PMAlgTrackMaker                        0.000914666     2.50395       57.2826      0.159683       9.02877        100    
reco:pmtracktccalo:Calorimetry                        6.4126e-05    0.00537936     0.0959116    0.000653167    0.0153913       100    
reco:pmtracktcpid:Chi2ParticleID                      2.6344e-05    0.000201746   0.00169268    9.9943e-05    0.000287087      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000167972     1.30352       29.3283      0.206256       3.97157        100    
reco:pmtrajfittccalo:Calorimetry                      5.5838e-05     0.0178434     0.552741     0.000912236    0.0722663       100    
reco:pmtrajfittcpid:Chi2ParticleID                    2.5313e-05    0.000296629   0.00359648    0.000110042   0.000579424      100    
reco:blurredcluster:BlurredClustering                 0.00719163     0.960975       8.08099      0.507675       1.40917        100    
reco:emtrkmichelid:EmTrackMichelId                     0.225295       10.3202       147.896       2.96838       23.3823        100    
reco:emshower:EMShower                                0.000233265    0.0234082     0.264755     0.00345177     0.0422257       100    
reco:cvnmap:CVNMapper                                 2.0654e-05     0.0130968     0.0473866     0.0117844     0.011169        100    
reco:cvneva:CVNEvaluator                              1.4634e-05     0.480468       1.3675       0.650522      0.315557        100    
reco:energyrecnumu:EnergyReco                         0.000249419    0.0316292     0.784523      0.0037212     0.109494        100    
reco:energyrecnue:EnergyReco                          0.000120641   0.00182574     0.0355494    0.00044979    0.00511359       100    
reco:energyrecnc:EnergyReco                           0.00170233    0.00344879     0.0389729    0.00203881    0.00536208       100    
reco:energyrecnumurange:EnergyReco                    0.000174237    0.0297911     0.778989     0.00178445     0.108668        100    
reco:energyrecnumumcs:EnergyReco                      0.000875154    0.0309961     0.780634     0.00215388     0.108604        100    
reco:opdec:Deconvolution                               0.0366303     0.310504       2.53318      0.184786      0.404781        100    
reco:ophitspe:OpHitFinderDeco                          0.229263       1.99069       4.74203       2.00312      0.887186        100    
reco:opflash:OpFlashFinder                            0.000203553   0.00106921    0.00286075     0.0010534    0.000496155      100    
reco:rns:RandomNumberSaver                            1.6544e-05    2.6817e-05    0.000237786    2.163e-05    2.23311e-05      100    
reco:anglereconue:NuAngularReco                       0.000137115   0.00140357     0.0325085    0.000297013   0.00447956       100    
reco:anglereconumu:NuAngularReco                       6.02e-05      0.0012408     0.0317792    0.000175667   0.00437381       100    
reco:anglereconuepfps:NuAngularReco                   0.00010066    0.00134216     0.0318705    0.000269833   0.00437823       100    
reco:anglereconumupfps:NuAngularReco                   8.459e-05    0.00129334     0.0313562    0.00023814    0.00434469       100    
[art]:TriggerResults:TriggerResultInserter             8.356e-06    9.79353e-06   3.6159e-05    9.2755e-06    2.98735e-06      100    
end_path:out1:RootOutput                               2.284e-06    3.81695e-06   1.4309e-05     3.016e-06    1.66651e-06      100    
end_path:out1:RootOutput(write)                        0.0165781     0.191523       0.82802      0.195533       0.14076        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 10688.1 MB
  Peak resident set size usage (VmHWM): 2368.83 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_597_20231207T160917Z_gen_g4_detsim_hitreco__20240510T031533Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_597_20231207T160917Z_gen_g4_detsim_hitreco__20240510T031533Z_reco2_reco_2024-11-22T_223431Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74568471_597_20231207T160917Z_gen_g4_detsim_hitreco__20240510T031533Z_reco2_reco_data_2024-11-22T_223431Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work