justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 422858.48@justin-prod-sched01.dune.hep.ac.uk

Jobsub ID422858.48@justin-prod-sched01.dune.hep.ac.uk
Workflow ID7961
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-06-26 20:46:40
SiteCZ_FZU
EntryNova_CZ_FZU_osgce1
Last heartbeat2025-06-26 21:12:34
From worker nodeHostnamemilan04.farm.particle.cz
cpuinfoAMD EPYC 7543 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-26 20:48:54
Input filesfardet-hd:nu_dune10kt_1x2x6_1408_620_20230826T093705Z_gen_g4_detsim_hitreco__20240229T185155Z_reco2.root
JobscriptExit code0
Real time4m (258s)
CPU time4m (243s = 94%)
Max RSS bytes1182720000 (1127 MiB)
Outputting started2025-06-26 20:53:13
Output files
Finished2025-06-26 21:12:34
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

e31a39388b50aa48e26ed/localProducts_larsoft_v10_08_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/ad/c6/nu_dune10kt_1x2x6_1408_620_20230826T093705Z_gen_g4_detsim_hitreco__20240229T185155Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 41st record. run: 1408 subRun: 1 event: 62041 at 26-Jun-2025 22:51:02 CEST
Begin processing the 42nd record. run: 1408 subRun: 1 event: 62042 at 26-Jun-2025 22:51:02 CEST
Begin processing the 43rd record. run: 1408 subRun: 1 event: 62043 at 26-Jun-2025 22:51:04 CEST
Begin processing the 44th record. run: 1408 subRun: 1 event: 62044 at 26-Jun-2025 22:51:05 CEST
Begin processing the 45th record. run: 1408 subRun: 1 event: 62045 at 26-Jun-2025 22:51:06 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 46th record. run: 1408 subRun: 1 event: 62046 at 26-Jun-2025 22:51:08 CEST
Begin processing the 47th record. run: 1408 subRun: 1 event: 62047 at 26-Jun-2025 22:51:09 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 48th record. run: 1408 subRun: 1 event: 62048 at 26-Jun-2025 22:51:10 CEST
Begin processing the 49th record. run: 1408 subRun: 1 event: 62049 at 26-Jun-2025 22:51:11 CEST
Begin processing the 50th record. run: 1408 subRun: 1 event: 62050 at 26-Jun-2025 22:51:12 CEST
Begin processing the 51st record. run: 1408 subRun: 1 event: 62051 at 26-Jun-2025 22:51:14 CEST
Begin processing the 52nd record. run: 1408 subRun: 1 event: 62052 at 26-Jun-2025 22:51:15 CEST
Begin processing the 53rd record. run: 1408 subRun: 1 event: 62053 at 26-Jun-2025 22:51:16 CEST
Begin processing the 54th record. run: 1408 subRun: 1 event: 62054 at 26-Jun-2025 22:51:17 CEST
Begin processing the 55th record. run: 1408 subRun: 1 event: 62055 at 26-Jun-2025 22:51:18 CEST
Begin processing the 56th record. run: 1408 subRun: 1 event: 62056 at 26-Jun-2025 22:51:20 CEST
Begin processing the 57th record. run: 1408 subRun: 1 event: 62057 at 26-Jun-2025 22:51:22 CEST
Begin processing the 58th record. run: 1408 subRun: 1 event: 62058 at 26-Jun-2025 22:51:23 CEST
Begin processing the 59th record. run: 1408 subRun: 1 event: 62059 at 26-Jun-2025 22:51:25 CEST
Begin processing the 60th record. run: 1408 subRun: 1 event: 62060 at 26-Jun-2025 22:51:27 CEST
Begin processing the 61st record. run: 1408 subRun: 1 event: 62061 at 26-Jun-2025 22:51:28 CEST
Begin processing the 62nd record. run: 1408 subRun: 1 event: 62062 at 26-Jun-2025 22:51:29 CEST
Begin processing the 63rd record. run: 1408 subRun: 1 event: 62063 at 26-Jun-2025 22:51:31 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 64th record. run: 1408 subRun: 1 event: 62064 at 26-Jun-2025 22:51:32 CEST
Begin processing the 65th record. run: 1408 subRun: 1 event: 62065 at 26-Jun-2025 22:51:34 CEST
Begin processing the 66th record. run: 1408 subRun: 1 event: 62066 at 26-Jun-2025 22:51:35 CEST
Begin processing the 67th record. run: 1408 subRun: 1 event: 62067 at 26-Jun-2025 22:51:36 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 68th record. run: 1408 subRun: 1 event: 62068 at 26-Jun-2025 22:51:37 CEST
Begin processing the 69th record. run: 1408 subRun: 1 event: 62069 at 26-Jun-2025 22:51:38 CEST
Begin processing the 70th record. run: 1408 subRun: 1 event: 62070 at 26-Jun-2025 22:51:39 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 71st record. run: 1408 subRun: 1 event: 62071 at 26-Jun-2025 22:51:40 CEST
Begin processing the 72nd record. run: 1408 subRun: 1 event: 62072 at 26-Jun-2025 22:51:51 CEST
Begin processing the 73rd record. run: 1408 subRun: 1 event: 62073 at 26-Jun-2025 22:51:53 CEST
Begin processing the 74th record. run: 1408 subRun: 1 event: 62074 at 26-Jun-2025 22:51:54 CEST
Begin processing the 75th record. run: 1408 subRun: 1 event: 62075 at 26-Jun-2025 22:51:55 CEST
Begin processing the 76th record. run: 1408 subRun: 1 event: 62076 at 26-Jun-2025 22:51:56 CEST
Begin processing the 77th record. run: 1408 subRun: 1 event: 62077 at 26-Jun-2025 22:51:57 CEST
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetVolumeIdToHitListMap
    in file:     /exp/dune/app/users/imawby/dunesw_sara/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
    in function: Run
    in file:     /exp/dune/app/users/imawby/dunesw_sara/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0002, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Begin processing the 78th record. run: 1408 subRun: 1 event: 62078 at 26-Jun-2025 22:51:57 CEST
Begin processing the 79th record. run: 1408 subRun: 1 event: 62079 at 26-Jun-2025 22:51:58 CEST
Begin processing the 80th record. run: 1408 subRun: 1 event: 62080 at 26-Jun-2025 22:51:59 CEST
Begin processing the 81st record. run: 1408 subRun: 1 event: 62081 at 26-Jun-2025 22:52:03 CEST
Begin processing the 82nd record. run: 1408 subRun: 1 event: 62082 at 26-Jun-2025 22:52:12 CEST
Begin processing the 83rd record. run: 1408 subRun: 1 event: 62083 at 26-Jun-2025 22:52:21 CEST
Begin processing the 84th record. run: 1408 subRun: 1 event: 62084 at 26-Jun-2025 22:52:23 CEST
Begin processing the 85th record. run: 1408 subRun: 1 event: 62085 at 26-Jun-2025 22:52:29 CEST
Begin processing the 86th record. run: 1408 subRun: 1 event: 62086 at 26-Jun-2025 22:52:30 CEST
Begin processing the 87th record. run: 1408 subRun: 1 event: 62087 at 26-Jun-2025 22:52:34 CEST
Begin processing the 88th record. run: 1408 subRun: 1 event: 62088 at 26-Jun-2025 22:52:40 CEST
Begin processing the 89th record. run: 1408 subRun: 1 event: 62089 at 26-Jun-2025 22:52:41 CEST
Begin processing the 90th record. run: 1408 subRun: 1 event: 62090 at 26-Jun-2025 22:52:47 CEST
Begin processing the 91st record. run: 1408 subRun: 1 event: 62091 at 26-Jun-2025 22:52:49 CEST
Begin processing the 92nd record. run: 1408 subRun: 1 event: 62092 at 26-Jun-2025 22:52:50 CEST
Begin processing the 93rd record. run: 1408 subRun: 1 event: 62093 at 26-Jun-2025 22:52:52 CEST
Begin processing the 94th record. run: 1408 subRun: 1 event: 62094 at 26-Jun-2025 22:52:54 CEST
Begin processing the 95th record. run: 1408 subRun: 1 event: 62095 at 26-Jun-2025 22:52:56 CEST
Begin processing the 96th record. run: 1408 subRun: 1 event: 62096 at 26-Jun-2025 22:53:02 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 97th record. run: 1408 subRun: 1 event: 62097 at 26-Jun-2025 22:53:04 CEST
Begin processing the 98th record. run: 1408 subRun: 1 event: 62098 at 26-Jun-2025 22:53:07 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 99th record. run: 1408 subRun: 1 event: 62099 at 26-Jun-2025 22:53:09 CEST
Begin processing the 100th record. run: 1408 subRun: 1 event: 62100 at 26-Jun-2025 22:53:11 CEST
26-Jun-2025 22:53:13 CEST  Closed output file "nu_dune10kt_1x2x6_1408_620_20230826T093705Z_gen_g4_detsim_hitreco__20240229T185155Z_reco2_reco2.root"
26-Jun-2025 22:53:13 CEST  Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/ad/c6/nu_dune10kt_1x2x6_1408_620_20230826T093705Z_gen_g4_detsim_hitreco__20240229T185155Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.0123736      1.94494       11.4362       1.28598       2.01661        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00106334     0.0138904     0.0650594     0.0139369     0.011612        100    
reco:pandora:StandardPandora                  0.00956157      1.92094       11.3972       1.27221       2.0071         100    
[art]:TriggerResults:TriggerResultInserter     1.012e-05    2.38459e-05   0.000280599   1.5785e-05    3.86625e-05      100    
end_path:out1:RootOutput                       2.68e-06     4.19839e-06    2.168e-05     3.565e-06    2.5545e-06       100    
end_path:out1:RootOutput(write)               0.000845238   0.00988869     0.0422273    0.00622484    0.00887721       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2185.04 MB
  Peak resident set size usage (VmHWM): 1182.72 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2025-08-15 01:24:52 UTC       justIN version: 01.03.02