justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 230488.12@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID230488.12@justin-prod-sched02.dune.hep.ac.uk
Workflow ID7961
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-06-26 20:49:09
SiteCZ_FZU
EntryNova_CZ_FZU_osgce1
Last heartbeat2025-06-26 21:22:23
From worker nodeHostnamemahagon24.farm.particle.cz
cpuinfoAMD EPYC 7301 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-26 20:49:49
Input filesfardet-hd:nu_dune10kt_1x2x6_1412_307_20230826T153307Z_gen_g4_detsim_hitreco__20240229T163517Z_reco2.root
JobscriptExit code0
Real time11m (717s)
CPU time9m (566s = 78%)
Max RSS bytes1213902848 (1157 MiB)
Outputting started2025-06-26 21:01:47
Output files
Finished2025-06-26 21:22:23
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/c0/d9/nu_dune10kt_1x2x6_1412_307_20230826T153307Z_gen_g4_detsim_hitreco__20240229T163517Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 38th record. run: 1412 subRun: 1 event: 30738 at 26-Jun-2025 22:55:27 CEST
Begin processing the 39th record. run: 1412 subRun: 1 event: 30739 at 26-Jun-2025 22:55:30 CEST
Begin processing the 40th record. run: 1412 subRun: 1 event: 30740 at 26-Jun-2025 22:55:35 CEST
Begin processing the 41st record. run: 1412 subRun: 1 event: 30741 at 26-Jun-2025 22:55:38 CEST
Begin processing the 42nd record. run: 1412 subRun: 1 event: 30742 at 26-Jun-2025 22:55:42 CEST
Begin processing the 43rd record. run: 1412 subRun: 1 event: 30743 at 26-Jun-2025 22:55:48 CEST
Begin processing the 44th record. run: 1412 subRun: 1 event: 30744 at 26-Jun-2025 22:55:52 CEST
Begin processing the 45th record. run: 1412 subRun: 1 event: 30745 at 26-Jun-2025 22:55:55 CEST
PandoraContentApi::GetList(*this, m_inputHitListName, pCaloHitList) return STATUS_CODE_NOT_INITIALIZED
    in function: GetVolumeIdToHitListMap
    in file:     /exp/dune/app/users/imawby/dunesw_sara/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 271
this->GetVolumeIdToHitListMap(volumeIdToHitListMap) return STATUS_CODE_NOT_INITIALIZED
    in function: Run
    in file:     /exp/dune/app/users/imawby/dunesw_sara/srcs/larpandoracontent/larpandoracontent/LArControlFlow/MasterAlgorithm.cc line#: 165
iter->second->Run() throw STATUS_CODE_NOT_INITIALIZED
    in function: RunAlgorithm
    in file:     /scratch/workspace/build-larbase/BUILDTYPE/prof/QUAL/s131-e26/label1/swarm/label2/SLF7/build/pandora/v03_16_00l/src/pandora-v03-16-00/PandoraSDK-v03-04-01/src/Api/PandoraContentApiImpl.cc line#: 235
Failure in algorithm Alg0002, LArDLMaster, STATUS_CODE_NOT_INITIALIZED
Begin processing the 46th record. run: 1412 subRun: 1 event: 30746 at 26-Jun-2025 22:55:56 CEST
Begin processing the 47th record. run: 1412 subRun: 1 event: 30747 at 26-Jun-2025 22:55:59 CEST
Begin processing the 48th record. run: 1412 subRun: 1 event: 30748 at 26-Jun-2025 22:56:05 CEST
Begin processing the 49th record. run: 1412 subRun: 1 event: 30749 at 26-Jun-2025 22:56:22 CEST
Begin processing the 50th record. run: 1412 subRun: 1 event: 30750 at 26-Jun-2025 22:56:27 CEST
Begin processing the 51st record. run: 1412 subRun: 1 event: 30751 at 26-Jun-2025 22:56:31 CEST
Begin processing the 52nd record. run: 1412 subRun: 1 event: 30752 at 26-Jun-2025 22:56:34 CEST
Begin processing the 53rd record. run: 1412 subRun: 1 event: 30753 at 26-Jun-2025 22:56:38 CEST
Begin processing the 54th record. run: 1412 subRun: 1 event: 30754 at 26-Jun-2025 22:56:41 CEST
Begin processing the 55th record. run: 1412 subRun: 1 event: 30755 at 26-Jun-2025 22:56:45 CEST
Begin processing the 56th record. run: 1412 subRun: 1 event: 30756 at 26-Jun-2025 22:56:48 CEST
Begin processing the 57th record. run: 1412 subRun: 1 event: 30757 at 26-Jun-2025 22:57:08 CEST
Begin processing the 58th record. run: 1412 subRun: 1 event: 30758 at 26-Jun-2025 22:57:12 CEST
Begin processing the 59th record. run: 1412 subRun: 1 event: 30759 at 26-Jun-2025 22:57:15 CEST
Begin processing the 60th record. run: 1412 subRun: 1 event: 30760 at 26-Jun-2025 22:57:19 CEST
Begin processing the 61st record. run: 1412 subRun: 1 event: 30761 at 26-Jun-2025 22:57:23 CEST
Begin processing the 62nd record. run: 1412 subRun: 1 event: 30762 at 26-Jun-2025 22:57:26 CEST
Begin processing the 63rd record. run: 1412 subRun: 1 event: 30763 at 26-Jun-2025 22:57:30 CEST
Begin processing the 64th record. run: 1412 subRun: 1 event: 30764 at 26-Jun-2025 22:57:34 CEST
Begin processing the 65th record. run: 1412 subRun: 1 event: 30765 at 26-Jun-2025 22:58:17 CEST
Begin processing the 66th record. run: 1412 subRun: 1 event: 30766 at 26-Jun-2025 22:58:21 CEST
Begin processing the 67th record. run: 1412 subRun: 1 event: 30767 at 26-Jun-2025 22:58:31 CEST
Begin processing the 68th record. run: 1412 subRun: 1 event: 30768 at 26-Jun-2025 22:58:35 CEST
Begin processing the 69th record. run: 1412 subRun: 1 event: 30769 at 26-Jun-2025 22:58:39 CEST
Begin processing the 70th record. run: 1412 subRun: 1 event: 30770 at 26-Jun-2025 22:58:43 CEST
Begin processing the 71st record. run: 1412 subRun: 1 event: 30771 at 26-Jun-2025 22:58:47 CEST
Begin processing the 72nd record. run: 1412 subRun: 1 event: 30772 at 26-Jun-2025 22:58:50 CEST
Begin processing the 73rd record. run: 1412 subRun: 1 event: 30773 at 26-Jun-2025 22:58:54 CEST
Begin processing the 74th record. run: 1412 subRun: 1 event: 30774 at 26-Jun-2025 22:58:57 CEST
Begin processing the 75th record. run: 1412 subRun: 1 event: 30775 at 26-Jun-2025 22:59:10 CEST
Begin processing the 76th record. run: 1412 subRun: 1 event: 30776 at 26-Jun-2025 22:59:14 CEST
Begin processing the 77th record. run: 1412 subRun: 1 event: 30777 at 26-Jun-2025 22:59:18 CEST
Begin processing the 78th record. run: 1412 subRun: 1 event: 30778 at 26-Jun-2025 22:59:27 CEST
Begin processing the 79th record. run: 1412 subRun: 1 event: 30779 at 26-Jun-2025 22:59:31 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 80th record. run: 1412 subRun: 1 event: 30780 at 26-Jun-2025 22:59:34 CEST
Begin processing the 81st record. run: 1412 subRun: 1 event: 30781 at 26-Jun-2025 22:59:38 CEST
Begin processing the 82nd record. run: 1412 subRun: 1 event: 30782 at 26-Jun-2025 22:59:45 CEST
Begin processing the 83rd record. run: 1412 subRun: 1 event: 30783 at 26-Jun-2025 22:59:48 CEST
Begin processing the 84th record. run: 1412 subRun: 1 event: 30784 at 26-Jun-2025 22:59:53 CEST
Begin processing the 85th record. run: 1412 subRun: 1 event: 30785 at 26-Jun-2025 22:59:57 CEST
Begin processing the 86th record. run: 1412 subRun: 1 event: 30786 at 26-Jun-2025 23:00:08 CEST
Begin processing the 87th record. run: 1412 subRun: 1 event: 30787 at 26-Jun-2025 23:00:11 CEST
Begin processing the 88th record. run: 1412 subRun: 1 event: 30788 at 26-Jun-2025 23:00:15 CEST
Begin processing the 89th record. run: 1412 subRun: 1 event: 30789 at 26-Jun-2025 23:00:59 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 90th record. run: 1412 subRun: 1 event: 30790 at 26-Jun-2025 23:01:03 CEST
Begin processing the 91st record. run: 1412 subRun: 1 event: 30791 at 26-Jun-2025 23:01:07 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 92nd record. run: 1412 subRun: 1 event: 30792 at 26-Jun-2025 23:01:11 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 93rd record. run: 1412 subRun: 1 event: 30793 at 26-Jun-2025 23:01:14 CEST
Begin processing the 94th record. run: 1412 subRun: 1 event: 30794 at 26-Jun-2025 23:01:17 CEST
Begin processing the 95th record. run: 1412 subRun: 1 event: 30795 at 26-Jun-2025 23:01:20 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 96th record. run: 1412 subRun: 1 event: 30796 at 26-Jun-2025 23:01:23 CEST
Begin processing the 97th record. run: 1412 subRun: 1 event: 30797 at 26-Jun-2025 23:01:27 CEST
Begin processing the 98th record. run: 1412 subRun: 1 event: 30798 at 26-Jun-2025 23:01:31 CEST
Begin processing the 99th record. run: 1412 subRun: 1 event: 30799 at 26-Jun-2025 23:01:35 CEST
Begin processing the 100th record. run: 1412 subRun: 1 event: 30800 at 26-Jun-2025 23:01:38 CEST
26-Jun-2025 23:01:46 CEST  Closed output file "nu_dune10kt_1x2x6_1412_307_20230826T153307Z_gen_g4_detsim_hitreco__20240229T163517Z_reco2_reco2.root"
26-Jun-2025 23:01:46 CEST  Closed input file "root://dune.dcache.nikhef.nl:1094/pnfs/nikhef.nl/data/dune/generic/rucio/fardet-hd/c0/d9/nu_dune10kt_1x2x6_1412_307_20230826T153307Z_gen_g4_detsim_hitreco__20240229T163517Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.148077       4.97683       42.5881       3.43927       5.92161        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0282981     0.0521248     0.221802      0.0489147     0.0197151       100    
reco:pandora:StandardPandora                   0.109142       4.89951       42.4206       3.35465       5.90622        100    
[art]:TriggerResults:TriggerResultInserter     1.622e-05    0.000174123    0.0111985    2.78675e-05   0.00112207       100    
end_path:out1:RootOutput                       3.116e-06    6.24101e-06   2.8514e-05    6.0765e-06    3.23078e-06      100    
end_path:out1:RootOutput(write)               0.00155599     0.0247348     0.118735      0.0183153     0.0225465       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2205.82 MB
  Peak resident set size usage (VmHWM): 1213.9 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2025-08-15 01:26:26 UTC       justIN version: 01.03.02