Jobsub ID 301665.132@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID	301665.132@justin-prod-sched01.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 03:06:47
Site	UK_Brunel
Entry	CMSHTPC_T2_UK_London_Brunel_dc2_22
Last heartbeat	2024-11-23 03:50:56
From worker node	Hostname	wn-b8-26-00.brunel.ac.uk
	cpuinfo	Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 03:10:36
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74548636_648_20231207T073603Z_gen_g4_detsim_hitreco__20240510T063853Z_reco2.root
Jobscript	Exit code	0
	Real time	40m (2404s)
	CPU time	35m (2154s = 89%)
Outputting started	2024-11-23 03:50:41
Output files
Finished	2024-11-23 03:50:56
List job events Wrapper job log
Jobscript log (last 10,000 characters)

99
Comparing two wires in the same plane: return failure
%MSG
Boundary wire vector sizes: 407, 342, 376
minwire 0: 1617
minwire 1: 1520
minwire 2: 1167
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Used alternate method to get min and max tdcs due to vertex determination failure: 0, 199
Classifier summary: 
Output 0: 0.254179, 0.650058, 0.0957637, 
Output 1: 0.166416, 0.606694, 0.212818, 0.0140723, 
Output 2: 0.457167, 0.542833, 

676
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 2735!
Begin processing the 100th record. run: 74548636 subRun: 1 event: 64900 at 23-Nov-2024 03:50:34 GMT
> Running Algorithm: Alg0001, LArPreProcessing
> Running Algorithm: Alg0002, LArDLMaster
> Running Algorithm: Alg0003, LArNeutrinoEventValidation
---RAW-MATCHING-OUTPUT--------------------------------------------------------------------------
Failure in algorithm Alg0003, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
%MSG-e ShowerProducedPtrsHolder:   LArPandoraModularShowerCreation:pandoraShower@BeginModule  23-Nov-2024 03:50:36 GMT run: 74548636 subRun: 1 event: 64900
Trying to add data product: InitialTrack. This element does not exist in the element holder
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrackMaker:pmtracktc@BeginModule  23-Nov-2024 03:50:36 GMT run: 74548636 subRun: 1 event: 64900
Comparing two wires in the same plane: return failure
%MSG
%MSG-e WireIDIntersectionCheck:  PMAlgTrajFitter:pmtrajfittc@BeginModule  23-Nov-2024 03:50:36 GMT run: 74548636 subRun: 1 event: 64900
Comparing two wires in the same plane: return failure
%MSG
302
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 837!
23-Nov-2024 03:50:40 GMT  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74548636_648_20231207T073603Z_gen_g4_detsim_hitreco__20240510T063853Z_reco2_reco_data_2024-11-23T_031039Z.root"
23-Nov-2024 03:50:40 GMT  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/b3/2d/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74548636_648_20231207T073603Z_gen_g4_detsim_hitreco__20240510T063853Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.711231       20.8318       554.615       7.68767       57.0705        100    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0225049     0.0525263     0.306979      0.0517039     0.0326758       100    
reco:linecluster:LineCluster                           0.127836      0.329269       7.42221      0.230532      0.725446        100    
reco:trajcluster:TrajCluster                           0.064169       1.2524        102.85       0.0704895      10.2243        100    
reco:pandora:StandardPandora                           0.0715173      1.96306       38.9349       1.51092       3.76515        100    
reco:pandoraTrack:LArPandoraTrackCreation             0.000150047   0.00436954     0.136517     0.00115461     0.0141471       100    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000200408    0.0102673     0.162476     0.00245144     0.0255295       100    
reco:pandoracalo:Calorimetry                          0.000114283    0.0132397     0.736175     0.00084322     0.0744186       100    
reco:pandorapid:Chi2ParticleID                        4.9827e-05    0.000169419   0.00107847     0.0001419    0.000150645      100    
reco:trkshowersplit:TrackShowerHits                   6.7141e-05     0.0118436     0.695258     0.000409132    0.0704338       100    
reco:pmtrack:PMAlgTrackMaker                          0.00136102      1.51396       64.0997       0.11022       6.58679        100    
reco:pmtrackcalo:Calorimetry                          9.2694e-05    0.00897696     0.556496     0.000676747    0.0555043       100    
reco:pmtrackpid:Chi2ParticleID                         3.836e-05    0.000229748    0.0038033    0.000131695   0.000420191      100    
reco:pmtrajfit:PMAlgTrajFitter                        0.000216468     1.15389       29.6724      0.220155       3.31154        100    
reco:pmtrajfitcalo:Calorimetry                        8.7992e-05     0.0140946     0.760786     0.00100066     0.0770654       100    
reco:pmtrajfitpid:Chi2ParticleID                      3.8661e-05    0.000170365   0.000833242   0.00013078    0.00013289       100    
reco:pmtracktc:PMAlgTrackMaker                        0.00124709      1.97268       96.8516      0.161597       9.82613        100    
reco:pmtracktccalo:Calorimetry                        9.2256e-05    0.00617936     0.302094     0.000596808    0.0305933       100    
reco:pmtracktcpid:Chi2ParticleID                      3.8123e-05    0.000213702   0.00244579    0.00012952    0.000280391      100    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000208951     1.03436       28.0938      0.203305       3.05265        100    
reco:pmtrajfittccalo:Calorimetry                      8.6784e-05     0.0115563     0.617905     0.000748597    0.0623955       100    
reco:pmtrajfittcpid:Chi2ParticleID                    3.8049e-05    0.000282955   0.00368459    0.000137282   0.000462547      100    
reco:blurredcluster:BlurredClustering                 8.9329e-05     0.836151       8.2366       0.440845       1.20381        100    
reco:emtrkmichelid:EmTrackMichelId                    0.000132099     7.99687       167.094       1.79159       18.8603        100    
reco:emshower:EMShower                                0.000190647    0.0234934     0.261297     0.00347338     0.0487507       100    
reco:cvnmap:CVNMapper                                 2.3628e-05     0.0121324     0.0468992     0.0103174     0.0114254       100    
reco:cvneva:CVNEvaluator                              1.9758e-05     0.537973      0.923915      0.789444      0.396264        100    
reco:energyrecnumu:EnergyReco                         0.000362406    0.0260288     0.537615     0.00401099     0.0870629       100    
reco:energyrecnue:EnergyReco                          0.000183331    0.0016666      0.04621     0.000601303   0.00478818       100    
reco:energyrecnc:EnergyReco                            0.0023929    0.00393922     0.0480205    0.00286718    0.00477324       100    
reco:energyrecnumurange:EnergyReco                    0.000224736    0.0241316     0.533579     0.00261376     0.085118        100    
reco:energyrecnumumcs:EnergyReco                       0.0023592     0.0257062     0.532559     0.00339944     0.0854205       100    
reco:opdec:Deconvolution                               0.0655556     0.331813      0.634409      0.334762       0.10726        100    
reco:ophitspe:OpHitFinderDeco                          0.191933       1.42958       2.77554       1.44662      0.579512        100    
reco:opflash:OpFlashFinder                            0.000119958    0.001442     0.00367374    0.00144955    0.000669124      100    
reco:rns:RandomNumberSaver                            1.9388e-05    3.51216e-05   0.000325869   3.1216e-05    3.10975e-05      100    
reco:anglereconue:NuAngularReco                       0.000149843   0.00110751     0.0354518    0.000351844   0.00367293       100    
reco:anglereconumu:NuAngularReco                      9.0276e-05    0.000973729    0.0359103    0.000211364   0.00372093       100    
reco:anglereconuepfps:NuAngularReco                   0.000143581   0.00111894     0.0356158    0.00036795    0.00368611       100    
reco:anglereconumupfps:NuAngularReco                  0.000130534    0.0010618     0.0366641    0.000277607   0.00379048       100    
[art]:TriggerResults:TriggerResultInserter             1.217e-05    1.48757e-05   4.9171e-05    1.35735e-05   4.42702e-06      100    
end_path:out1:RootOutput                               3.127e-06    6.5662e-06    2.1046e-05     6.289e-06    3.77711e-06      100    
end_path:out1:RootOutput(write)                       0.00877732     0.220795       1.11036      0.247938       0.15839        100    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 8343.39 MB
  Peak resident set size usage (VmHWM): 2242.34 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74548636_648_20231207T073603Z_gen_g4_detsim_hitreco__20240510T063853Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74548636_648_20231207T073603Z_gen_g4_detsim_hitreco__20240510T063853Z_reco2_reco_2024-11-23T_031039Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_74548636_648_20231207T073603Z_gen_g4_detsim_hitreco__20240510T063853Z_reco2_reco_data_2024-11-23T_031039Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work