Jobsub ID 108683.77@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108683.77@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 02:04:29
Site	UK_Lancaster
Entry	UBoone_UK_Lancaster_HEC_grendel_ce02
Last heartbeat	2024-11-23 03:44:41
From worker node	Hostname	comp23-13
	cpuinfo	Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	257400 (71 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 02:22:55
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6398928_715_20231202T065642Z_gen_g4_detsim_hitreco__20240507T204151Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (4892s)
	CPU time	1h (4616s = 94%)
Outputting started	2024-11-23 03:44:27
Output files
Finished	2024-11-23 03:44:41
List job events Wrapper job log
Jobscript log (last 10,000 characters)

res in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 03:44:22 GMT run: 6398928 subRun: 1 event: 71600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtrack@BeginModule  23-Nov-2024 03:44:22 GMT run: 6398928 subRun: 1 event: 71600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 03:44:22 GMT run: 6398928 subRun: 1 event: 71600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfit@BeginModule  23-Nov-2024 03:44:22 GMT run: 6398928 subRun: 1 event: 71600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 03:44:22 GMT run: 6398928 subRun: 1 event: 71600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 03:44:22 GMT run: 6398928 subRun: 1 event: 71600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 03:44:22 GMT run: 6398928 subRun: 1 event: 71600
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 03:44:22 GMT run: 6398928 subRun: 1 event: 71600
1st wire C:0 T:11 P:2 W:686 does not exist (max wire number: 480)
%MSG
%MSG-e WireIDIntersectionCheck:  EMShower:emshower@BeginModule  23-Nov-2024 03:44:23 GMT run: 6398928 subRun: 1 event: 71600
Comparing two wires in the same plane: return failure
%MSG
419
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 1250!
23-Nov-2024 03:44:26 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6398928_715_20231202T065642Z_gen_g4_detsim_hitreco__20240507T204151Z_reco2_reco_data_2024-11-23T_022258Z.root"
23-Nov-2024 03:44:26 GMT  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/10/58/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6398928_715_20231202T065642Z_gen_g4_detsim_hitreco__20240507T204151Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                              2.36912       46.7131       725.905       13.2766       89.709         100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.012026      0.122168      0.637335      0.0910786     0.101533        100    
reco:linecluster:LineCluster                           0.133258      0.543191       10.2239      0.379156       1.02792        100    
reco:trajcluster:TrajCluster                           0.0350709      1.47814       33.4159      0.149793       4.2066         100    
reco:pandora:StandardPandora                            1.0937        4.85859       264.207       1.43077       26.1687        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000148364    0.0125617     0.198349     0.00255712     0.0272844       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000182044    0.0218242     0.542258     0.00417307     0.0652784       100    
reco:pandoracalo:Calorimetry                          0.000100671    0.0440392      1.93569     0.00146167     0.200242        100    
reco:pandorapid:Chi2ParticleID                        3.6806e-05    0.000195385   0.00124174    0.000152823   0.000164958      100    
reco:trkshowersplit:TrackShowerHits                   0.00010083     0.0312069     0.786422     0.00122092     0.0919863       100    
reco:pmtrack:PMAlgTrackMaker                          0.00120112      3.19684       56.9848       0.22183       7.55532        100    
reco:pmtrackcalo:Calorimetry                          8.7002e-05     0.0193802      0.35484     0.00125761     0.0528791       100    
reco:pmtrackpid:Chi2ParticleID                        3.0583e-05    0.000434163   0.00644354    0.000163197   0.000773444      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000198243     3.22263       61.7013      0.441201       8.77782        100    
reco:pmtrajfitcalo:Calorimetry                        6.6729e-05     0.0424187      1.62379     0.00200694     0.171563        100    
reco:pmtrajfitpid:Chi2ParticleID                      2.7682e-05    0.000220138   0.000891061   0.000152312   0.000179394      100    
reco:pmtracktc:PMAlgTrackMaker                        0.00109899      3.86188       54.5885      0.386872       8.79445        100    
reco:pmtracktccalo:Calorimetry                        8.0735e-05     0.0142465     0.238109     0.00123445     0.0365899       100    
reco:pmtracktcpid:Chi2ParticleID                      3.0231e-05    0.000328985   0.00209007    0.000155607   0.00040019       100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000179857     2.13378       32.9214      0.383421       4.37376        100    
reco:pmtrajfittccalo:Calorimetry                      6.4962e-05     0.029693      0.710647     0.00169626     0.0872089       100    
reco:pmtrajfittcpid:Chi2ParticleID                     2.618e-05    0.000465397   0.00401828    0.000182991   0.000639173      100    
reco:blurredcluster:BlurredClustering                 0.00425615      1.61155       10.3441      0.741037       2.04469        100    
reco:emtrkmichelid:EmTrackMichelId                     0.131377       21.3275       192.52        5.24263       34.392         100    
reco:emshower:EMShower                                0.00024906     0.0383624     0.403673     0.00810269     0.0660065       100    
reco:cvnmap:CVNMapper                                 1.9607e-05     0.017814      0.0479874     0.0169895     0.0112391       100    
reco:cvneva:CVNEvaluator                              1.5598e-05     0.679335      0.919556      0.825593      0.321458        100    
reco:energyrecnumu:EnergyReco                         0.000302835    0.0743495      1.91051     0.00649393     0.234515        100    
reco:energyrecnue:EnergyReco                          0.000139896   0.00391002     0.0628374    0.000793943   0.00857324       100    
reco:energyrecnc:EnergyReco                           0.00183127    0.00566304     0.0650503    0.00237531    0.00864991       100    
reco:energyrecnumurange:EnergyReco                    0.000240405    0.0710836      1.90788      0.0023171     0.232986        100    
reco:energyrecnumumcs:EnergyReco                      0.00183748     0.0723737      1.90517     0.00337448     0.232032        100    
reco:opdec:Deconvolution                               0.135282       0.43596       1.39375      0.413994      0.176521        100    
reco:ophitspe:OpHitFinderDeco                          0.562552       2.46639       4.29287       2.48341      0.899316        100    
reco:opflash:OpFlashFinder                            0.000372623   0.00145422    0.00374585    0.00142781    0.00062081       100    
reco:rns:RandomNumberSaver                            1.8325e-05    3.45478e-05   0.000256283   3.3065e-05    2.4513e-05       100    
reco:anglereconue:NuAngularReco                       0.000160927   0.00322247     0.0554805    0.000426986   0.00751292       100    
reco:anglereconumu:NuAngularReco                      7.1401e-05    0.00293682     0.0534066    0.000271055   0.00723883       100    
reco:anglereconuepfps:NuAngularReco                   0.000120848   0.00306472     0.0538321    0.000362452   0.00727433       100    
reco:anglereconumupfps:NuAngularReco                  0.00010235    0.00302471     0.0537639    0.000332985   0.00726616       100    
[art]:TriggerResults:TriggerResultInserter             8.456e-06    1.23256e-05   4.3623e-05    1.17465e-05   3.88611e-06      100    
end_path:out1:RootOutput                               2.372e-06    5.35064e-06   1.8257e-05     5.126e-06    2.87677e-06      100    
end_path:out1:RootOutput(write)                        0.0227486     0.258992       1.07436      0.223173      0.176544        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9646.08 MB
  Peak resident set size usage (VmHWM): 2489.18 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6398928_715_20231202T065642Z_gen_g4_detsim_hitreco__20240507T204151Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6398928_715_20231202T065642Z_gen_g4_detsim_hitreco__20240507T204151Z_reco2_reco_2024-11-23T_022258Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6398928_715_20231202T065642Z_gen_g4_detsim_hitreco__20240507T204151Z_reco2_reco_data_2024-11-23T_022258Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work