justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 230488.8@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID230488.8@justin-prod-sched02.dune.hep.ac.uk
Workflow ID7961
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-06-26 20:49:09
SiteCZ_FZU
EntryNova_CZ_FZU_osgce2
Last heartbeat2025-06-26 21:24:35
From worker nodeHostnamelupin33.farm.particle.cz
cpuinfoAMD EPYC 7532 32-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit172800 (48 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-26 20:49:47
Input filesfardet-hd:nu_dune10kt_1x2x6_1408_201_20230826T093340Z_gen_g4_detsim_hitreco__20240229T185222Z_reco2.root
JobscriptExit code0
Real time15m (910s)
CPU time12m (760s = 83%)
Max RSS bytes1423720448 (1357 MiB)
Outputting started2025-06-26 21:04:57
Output files
Finished2025-06-26 21:24:35
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

---------------------------------------------------
Input PFN = root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/6c/5f/nu_dune10kt_1x2x6_1408_201_20230826T093340Z_gen_g4_detsim_hitreco__20240229T185222Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
Begin processing the 29th record. run: 1408 subRun: 1 event: 20129 at 26-Jun-2025 23:01:22 CEST
Begin processing the 30th record. run: 1408 subRun: 1 event: 20130 at 26-Jun-2025 23:01:26 CEST
Begin processing the 31st record. run: 1408 subRun: 1 event: 20131 at 26-Jun-2025 23:01:28 CEST
Begin processing the 32nd record. run: 1408 subRun: 1 event: 20132 at 26-Jun-2025 23:01:30 CEST
Begin processing the 33rd record. run: 1408 subRun: 1 event: 20133 at 26-Jun-2025 23:01:33 CEST
Begin processing the 34th record. run: 1408 subRun: 1 event: 20134 at 26-Jun-2025 23:01:35 CEST
Begin processing the 35th record. run: 1408 subRun: 1 event: 20135 at 26-Jun-2025 23:01:37 CEST
Begin processing the 36th record. run: 1408 subRun: 1 event: 20136 at 26-Jun-2025 23:01:40 CEST
Begin processing the 37th record. run: 1408 subRun: 1 event: 20137 at 26-Jun-2025 23:01:44 CEST
Begin processing the 38th record. run: 1408 subRun: 1 event: 20138 at 26-Jun-2025 23:01:56 CEST
Begin processing the 39th record. run: 1408 subRun: 1 event: 20139 at 26-Jun-2025 23:01:59 CEST
Begin processing the 40th record. run: 1408 subRun: 1 event: 20140 at 26-Jun-2025 23:02:01 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 41st record. run: 1408 subRun: 1 event: 20141 at 26-Jun-2025 23:02:03 CEST
Begin processing the 42nd record. run: 1408 subRun: 1 event: 20142 at 26-Jun-2025 23:02:05 CEST
Begin processing the 43rd record. run: 1408 subRun: 1 event: 20143 at 26-Jun-2025 23:02:08 CEST
Begin processing the 44th record. run: 1408 subRun: 1 event: 20144 at 26-Jun-2025 23:02:11 CEST
Begin processing the 45th record. run: 1408 subRun: 1 event: 20145 at 26-Jun-2025 23:02:14 CEST
Begin processing the 46th record. run: 1408 subRun: 1 event: 20146 at 26-Jun-2025 23:02:16 CEST
Begin processing the 47th record. run: 1408 subRun: 1 event: 20147 at 26-Jun-2025 23:02:19 CEST
Begin processing the 48th record. run: 1408 subRun: 1 event: 20148 at 26-Jun-2025 23:02:21 CEST
Begin processing the 49th record. run: 1408 subRun: 1 event: 20149 at 26-Jun-2025 23:02:26 CEST
Begin processing the 50th record. run: 1408 subRun: 1 event: 20150 at 26-Jun-2025 23:02:28 CEST
Begin processing the 51st record. run: 1408 subRun: 1 event: 20151 at 26-Jun-2025 23:02:30 CEST
Begin processing the 52nd record. run: 1408 subRun: 1 event: 20152 at 26-Jun-2025 23:02:32 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 53rd record. run: 1408 subRun: 1 event: 20153 at 26-Jun-2025 23:02:36 CEST
Begin processing the 54th record. run: 1408 subRun: 1 event: 20154 at 26-Jun-2025 23:02:38 CEST
Begin processing the 55th record. run: 1408 subRun: 1 event: 20155 at 26-Jun-2025 23:02:42 CEST
Begin processing the 56th record. run: 1408 subRun: 1 event: 20156 at 26-Jun-2025 23:02:44 CEST
Begin processing the 57th record. run: 1408 subRun: 1 event: 20157 at 26-Jun-2025 23:02:46 CEST
Begin processing the 58th record. run: 1408 subRun: 1 event: 20158 at 26-Jun-2025 23:02:49 CEST
Begin processing the 59th record. run: 1408 subRun: 1 event: 20159 at 26-Jun-2025 23:02:54 CEST
Begin processing the 60th record. run: 1408 subRun: 1 event: 20160 at 26-Jun-2025 23:02:59 CEST
Begin processing the 61st record. run: 1408 subRun: 1 event: 20161 at 26-Jun-2025 23:03:01 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 62nd record. run: 1408 subRun: 1 event: 20162 at 26-Jun-2025 23:03:03 CEST
Begin processing the 63rd record. run: 1408 subRun: 1 event: 20163 at 26-Jun-2025 23:03:06 CEST
Begin processing the 64th record. run: 1408 subRun: 1 event: 20164 at 26-Jun-2025 23:03:10 CEST
Begin processing the 65th record. run: 1408 subRun: 1 event: 20165 at 26-Jun-2025 23:03:12 CEST
Begin processing the 66th record. run: 1408 subRun: 1 event: 20166 at 26-Jun-2025 23:03:15 CEST
Begin processing the 67th record. run: 1408 subRun: 1 event: 20167 at 26-Jun-2025 23:03:17 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 68th record. run: 1408 subRun: 1 event: 20168 at 26-Jun-2025 23:03:20 CEST
Begin processing the 69th record. run: 1408 subRun: 1 event: 20169 at 26-Jun-2025 23:03:22 CEST
Begin processing the 70th record. run: 1408 subRun: 1 event: 20170 at 26-Jun-2025 23:03:25 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 71st record. run: 1408 subRun: 1 event: 20171 at 26-Jun-2025 23:03:27 CEST
Begin processing the 72nd record. run: 1408 subRun: 1 event: 20172 at 26-Jun-2025 23:03:32 CEST
Begin processing the 73rd record. run: 1408 subRun: 1 event: 20173 at 26-Jun-2025 23:03:35 CEST
ElectronInitialRegionRefinementAlgorithm: unable to find shower pfo list ShowerParticles3D
Begin processing the 74th record. run: 1408 subRun: 1 event: 20174 at 26-Jun-2025 23:03:37 CEST
Begin processing the 75th record. run: 1408 subRun: 1 event: 20175 at 26-Jun-2025 23:03:40 CEST
Begin processing the 76th record. run: 1408 subRun: 1 event: 20176 at 26-Jun-2025 23:03:50 CEST
Begin processing the 77th record. run: 1408 subRun: 1 event: 20177 at 26-Jun-2025 23:03:53 CEST
Begin processing the 78th record. run: 1408 subRun: 1 event: 20178 at 26-Jun-2025 23:03:57 CEST
Begin processing the 79th record. run: 1408 subRun: 1 event: 20179 at 26-Jun-2025 23:04:00 CEST
Begin processing the 80th record. run: 1408 subRun: 1 event: 20180 at 26-Jun-2025 23:04:02 CEST
Begin processing the 81st record. run: 1408 subRun: 1 event: 20181 at 26-Jun-2025 23:04:05 CEST
Begin processing the 82nd record. run: 1408 subRun: 1 event: 20182 at 26-Jun-2025 23:04:08 CEST
Begin processing the 83rd record. run: 1408 subRun: 1 event: 20183 at 26-Jun-2025 23:04:10 CEST
Begin processing the 84th record. run: 1408 subRun: 1 event: 20184 at 26-Jun-2025 23:04:13 CEST
Begin processing the 85th record. run: 1408 subRun: 1 event: 20185 at 26-Jun-2025 23:04:15 CEST
Begin processing the 86th record. run: 1408 subRun: 1 event: 20186 at 26-Jun-2025 23:04:18 CEST
Begin processing the 87th record. run: 1408 subRun: 1 event: 20187 at 26-Jun-2025 23:04:21 CEST
Begin processing the 88th record. run: 1408 subRun: 1 event: 20188 at 26-Jun-2025 23:04:23 CEST
Begin processing the 89th record. run: 1408 subRun: 1 event: 20189 at 26-Jun-2025 23:04:26 CEST
Begin processing the 90th record. run: 1408 subRun: 1 event: 20190 at 26-Jun-2025 23:04:30 CEST
Begin processing the 91st record. run: 1408 subRun: 1 event: 20191 at 26-Jun-2025 23:04:32 CEST
Begin processing the 92nd record. run: 1408 subRun: 1 event: 20192 at 26-Jun-2025 23:04:34 CEST
Begin processing the 93rd record. run: 1408 subRun: 1 event: 20193 at 26-Jun-2025 23:04:37 CEST
Begin processing the 94th record. run: 1408 subRun: 1 event: 20194 at 26-Jun-2025 23:04:40 CEST
Begin processing the 95th record. run: 1408 subRun: 1 event: 20195 at 26-Jun-2025 23:04:42 CEST
Begin processing the 96th record. run: 1408 subRun: 1 event: 20196 at 26-Jun-2025 23:04:44 CEST
Begin processing the 97th record. run: 1408 subRun: 1 event: 20197 at 26-Jun-2025 23:04:46 CEST
Begin processing the 98th record. run: 1408 subRun: 1 event: 20198 at 26-Jun-2025 23:04:49 CEST
Begin processing the 99th record. run: 1408 subRun: 1 event: 20199 at 26-Jun-2025 23:04:51 CEST
Begin processing the 100th record. run: 1408 subRun: 1 event: 20200 at 26-Jun-2025 23:04:54 CEST
26-Jun-2025 23:04:56 CEST  Closed output file "nu_dune10kt_1x2x6_1408_201_20230826T093340Z_gen_g4_detsim_hitreco__20240229T185222Z_reco2_reco2.root"
26-Jun-2025 23:04:56 CEST  Closed input file "root://otter12.grid.surfsara.nl:21094/pnfs/grid.sara.nl/data/dune/disk/RSE/fardet-hd/6c/5f/nu_dune10kt_1x2x6_1408_201_20230826T093340Z_gen_g4_detsim_hitreco__20240229T185222Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      1.34662       7.26588       424.883       2.1283        42.4297        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.029402      0.0407186     0.304702      0.0332509     0.0277887       100    
reco:pandora:StandardPandora                    1.31284       7.21004       424.697       2.06936       42.4158        100    
[art]:TriggerResults:TriggerResultInserter     1.311e-05    2.49426e-05   9.0621e-05     2.307e-05    1.11975e-05      100    
end_path:out1:RootOutput                       2.99e-06     4.77201e-06    2.576e-05     4.03e-06     2.6947e-06       100    
end_path:out1:RootOutput(write)               0.00196072     0.0148446     0.130779      0.0101633     0.016701        100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2415.64 MB
  Peak resident set size usage (VmHWM): 1423.72 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2025-08-15 06:44:25 UTC       justIN version: 01.03.02