Jobsub ID 108743.17@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	108743.17@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	4213
Stage ID	1
User name	ismerio@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2024-11-23 04:34:30
Site	CERN
Entry	CMSHTPC_T2_CH_CERN_ce511
Last heartbeat	2024-11-23 06:23:47
From worker node	Hostname	b9p08p5323.cern.ch
	cpuinfo	AMD EPYC 7543 32-Core Processor
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	4194304000 (4000 MiB)
	Wall seconds limit	171000 (47 hours)
	Inner Apptainer?	True
Job state	outputting_failed
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2024-11-23 04:36:52
Input files	fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1794_495_20231028T044445Z_gen_g4_detsim_hitreco__20240503T051305Z_reco2.root
Jobscript	Exit code	0
	Real time	1h (6378s)
	CPU time	1h (6428s = 100%)
Outputting started	2024-11-23 06:23:12
Output files
Finished	2024-11-23 06:23:47
List job events Wrapper job log
Jobscript log (last 10,000 characters)

ing
PreProcessingAlgorithm: could not replace current calo hit list with list named: CaloHitList2D
> Running Algorithm: Alg0002, LArDLMaster
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetVolumeIdToHitListMap
    in file:     /exp/dune/app/users/ismerio/env-testreco/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
    in function: Run
    in file:     /exp/dune/app/users/ismerio/env-testreco/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0002, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
> Running Algorithm: Alg0003, LArNeutrinoEventValidation
PandoraContentApi::GetList(*this, m_caloHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
    in function: Run
    in file:     /exp/dune/app/users/ismerio/env-testreco/srcs/larpandoracontent/larpandoracontent/LArMonitoring/EventValidationBaseAlgorithm.cc line#: 64
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0003, LArNeutrinoEventValidation, STATUS_CODE_NOT_INITIALIZED
61
***Wiener Filter*******ApplyPostfilter***
Running Ophitfinder with InputDigiType = 'recob'
Found hits: 104!
23-Nov-2024 07:23:09 CET  Closed output file "atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1794_495_20231028T044445Z_gen_g4_detsim_hitreco__20240503T051305Z_reco2_reco_data_2024-11-23T_043658Z.root"
23-Nov-2024 07:23:09 CET  Closed input file "root://golias100.farm.particle.cz:1094/dpm/farm.particle.cz/home/dune/RSE/fardet-hd/46/fb/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1794_495_20231028T044445Z_gen_g4_detsim_hitreco__20240503T051305Z_reco2.root"

========================================================================================================================================
TimeTracker printout (sec)                                Min           Avg           Max         Median          RMS         nEvts   
========================================================================================================================================
Full event                                             0.586163       30.1416       613.957       9.50551       69.4953        200    
----------------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                                 0.0171522     0.0449814     0.171674      0.0442563     0.0172395       200    
reco:linecluster:LineCluster                           0.0943209     0.445851       14.1642      0.185546       1.50465        200    
reco:trajcluster:TrajCluster                           0.0477236     0.950575       41.1539      0.0598323      4.30977        200    
reco:pandora:StandardPandora                           0.0599749      3.22397       198.537       1.45278       14.5271        200    
reco:pandoraTrack:LArPandoraTrackCreation             0.000135401   0.00705247     0.0838054    0.00146011     0.0149314       200    
reco:pandoraShower:LArPandoraModularShowerCreation    0.000194051    0.0167221     0.457525     0.00335135     0.0439682       200    
reco:pandoracalo:Calorimetry                          0.000110531    0.0229766     0.857621     0.00100533     0.0900085       200    
reco:pandorapid:Chi2ParticleID                        4.1921e-05    0.000203947   0.00237143    0.000149156   0.000228453      200    
reco:trkshowersplit:TrackShowerHits                   7.8611e-05     0.0289634      1.31493     0.000599055    0.120875        200    
reco:pmtrack:PMAlgTrackMaker                           0.0011773      2.84117       67.2163      0.155674       9.27245        200    
reco:pmtrackcalo:Calorimetry                          7.5251e-05     0.0118149     0.467435     0.000909513    0.043978        200    
reco:pmtrackpid:Chi2ParticleID                         2.94e-05     0.000320367   0.00310308    0.000156561   0.000511378      200    
reco:pmtrajfit:PMAlgTrajFitter                        0.000187642     2.36606       57.4524      0.244424       7.48485        200    
reco:pmtrajfitcalo:Calorimetry                         6.927e-05     0.0229351     0.898654     0.00123428     0.0889586       200    
reco:pmtrajfitpid:Chi2ParticleID                       2.934e-05    0.000218762   0.00118465    0.000158206   0.00019918       200    
reco:pmtracktc:PMAlgTrackMaker                        0.000974648     3.25799       75.3945      0.147585       10.6676        200    
reco:pmtracktccalo:Calorimetry                        7.0451e-05    0.00808513     0.161478     0.000938228    0.0219776       200    
reco:pmtracktcpid:Chi2ParticleID                      2.8281e-05    0.000283052    0.0019974    0.000158287   0.00035165       200    
reco:pmtrajfittc:PMAlgTrajFitter                      0.000164132     1.76122       32.719       0.237224       4.53177        200    
reco:pmtrajfittccalo:Calorimetry                      6.7851e-05     0.0163319     0.408707     0.00101727     0.0548934       200    
reco:pmtrajfittcpid:Chi2ParticleID                     2.867e-05    0.000411114   0.00466455    0.000166967   0.000716444      200    
reco:blurredcluster:BlurredClustering                 0.000154301     1.11277       8.6704       0.472203       1.65464        200    
reco:emtrkmichelid:EmTrackMichelId                    0.000140561     10.2447       125.238       2.10135       20.3488        200    
reco:emshower:EMShower                                0.000216452    0.0299014     0.300412     0.00659762     0.0523399       200    
reco:cvnmap:CVNMapper                                 2.3541e-05     0.0110427     0.0439576     0.0123611    0.00981484       200    
reco:cvneva:CVNEvaluator                              1.7231e-05     0.696383       1.51735      0.943058      0.525881        200    
reco:energyrecnumu:EnergyReco                         0.000331523    0.0511947      4.23689     0.00446253     0.318825        200    
reco:energyrecnue:EnergyReco                          0.000143691   0.00250517     0.0414814    0.000618561   0.00506289       200    
reco:energyrecnc:EnergyReco                           0.00167552    0.00513207     0.0321813    0.00377864    0.00495845       200    
reco:energyrecnumurange:EnergyReco                    0.000146681    0.0507825      4.24164     0.00276622     0.321356        200    
reco:energyrecnumumcs:EnergyReco                      0.000772577    0.0524947      4.32133     0.00409683     0.326468        200    
reco:opdec:Deconvolution                               0.0500453     0.309303      0.598218      0.312456      0.112552        200    
reco:ophitspe:OpHitFinderDeco                          0.144736       2.27194       5.33912       2.11682       1.15381        200    
reco:opflash:OpFlashFinder                            0.000157272   0.00160489    0.00415894    0.00157253    0.000737619      200    
reco:rns:RandomNumberSaver                             1.991e-05    4.4694e-05    0.000409744    4.179e-05    2.97142e-05      200    
reco:anglereconue:NuAngularReco                       0.000167621   0.00180917     0.0291114    0.000446709   0.00417901       200    
reco:anglereconumu:NuAngularReco                      7.8661e-05    0.00158443     0.0287607    0.000218842   0.00416884       200    
reco:anglereconuepfps:NuAngularReco                   0.000133781   0.00174163     0.0286845    0.000389214   0.00415007       200    
reco:anglereconumupfps:NuAngularReco                  0.000112021   0.00166667     0.0285909    0.000303978   0.00416672       200    
[art]:TriggerResults:TriggerResultInserter             8.96e-06     1.52881e-05    5.342e-05    1.3935e-05    5.04551e-06      200    
end_path:out1:RootOutput                               3.38e-06     7.24268e-06    3.266e-05     7.135e-06    3.19824e-06      200    
end_path:out1:RootOutput(write)                        0.0116397     0.264541       1.20646       0.26066      0.214342        200    
========================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 21722.1 MB
  Peak resident set size usage (VmHWM): 2285.75 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
Validation_ccnc_atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1794_495_20231028T044445Z_gen_g4_detsim_hitreco__20240503T051305Z_reco2.root
all-input-dids.txt
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1794_495_20231028T044445Z_gen_g4_detsim_hitreco__20240503T051305Z_reco2_reco_2024-11-23T_043658Z.log
atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1794_495_20231028T044445Z_gen_g4_detsim_hitreco__20240503T051305Z_reco2_reco_data_2024-11-23T_043658Z.root
debugprod.log
jobscript.log
justin-processed-pfns.txt
reco2_hist.root
MyPandoraSettings_Master_Atmos_DUNEFD.xml
MyPandoraSettings_Master_DUNEFD.xml
build_slf7.x86_64
localProducts_larsoft_v09_91_02_e26_prof
setup_env-testreco.sh
srcs
temp.txt
temp2.txt
test
work