justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 226777.146@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID226777.146@justin-prod-sched02.dune.hep.ac.uk
Workflow ID7806
Stage ID1
User namehsouza@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes3145728000 (3000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-20 19:14:34
SiteFR_CCIN2P3
EntryDUNE_FR_CCIN2P3_cccondorce01
Last heartbeat2025-06-20 19:57:58
From worker nodeHostnameccwcondor0663
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes3145728000 (3000 MiB)
Wall seconds limit106200 (29 hours)
GPU
Inner Apptainer?True
Job stateoutputting_failed
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-20 19:27:44
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_130_20231027T174237Z_gen_g4_detsim_hitreco__20240503T043803Z_reco2.root
JobscriptExit code0
Real time6m (367s)
CPU time1m (79s = 21%)
Max RSS bytes1241088000 (1183 MiB)
Outputting started2025-06-20 19:33:51
Output files
Finished2025-06-20 19:57:58
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

28 at 20-Jun-2025 21:31:25 CEST
Begin processing the 129th record. run: 1793 subRun: 1 event: 26129 at 20-Jun-2025 21:31:26 CEST
Begin processing the 130th record. run: 1793 subRun: 1 event: 26130 at 20-Jun-2025 21:31:26 CEST
Begin processing the 131st record. run: 1793 subRun: 1 event: 26131 at 20-Jun-2025 21:31:28 CEST
Begin processing the 132nd record. run: 1793 subRun: 1 event: 26132 at 20-Jun-2025 21:31:29 CEST
Begin processing the 133rd record. run: 1793 subRun: 1 event: 26133 at 20-Jun-2025 21:31:30 CEST
Begin processing the 134th record. run: 1793 subRun: 1 event: 26134 at 20-Jun-2025 21:31:32 CEST
Begin processing the 135th record. run: 1793 subRun: 1 event: 26135 at 20-Jun-2025 21:31:34 CEST
Begin processing the 136th record. run: 1793 subRun: 1 event: 26136 at 20-Jun-2025 21:31:35 CEST
Begin processing the 137th record. run: 1793 subRun: 1 event: 26137 at 20-Jun-2025 21:31:36 CEST
Begin processing the 138th record. run: 1793 subRun: 1 event: 26138 at 20-Jun-2025 21:31:36 CEST
Begin processing the 139th record. run: 1793 subRun: 1 event: 26139 at 20-Jun-2025 21:31:37 CEST
Begin processing the 140th record. run: 1793 subRun: 1 event: 26140 at 20-Jun-2025 21:31:38 CEST
Begin processing the 141st record. run: 1793 subRun: 1 event: 26141 at 20-Jun-2025 21:31:40 CEST
Begin processing the 142nd record. run: 1793 subRun: 1 event: 26142 at 20-Jun-2025 21:31:41 CEST
Begin processing the 143rd record. run: 1793 subRun: 1 event: 26143 at 20-Jun-2025 21:31:44 CEST
Begin processing the 144th record. run: 1793 subRun: 1 event: 26144 at 20-Jun-2025 21:31:45 CEST
Begin processing the 145th record. run: 1793 subRun: 1 event: 26145 at 20-Jun-2025 21:31:46 CEST
Begin processing the 146th record. run: 1793 subRun: 1 event: 26146 at 20-Jun-2025 21:31:48 CEST
Begin processing the 147th record. run: 1793 subRun: 1 event: 26147 at 20-Jun-2025 21:31:50 CEST
Begin processing the 148th record. run: 1793 subRun: 1 event: 26148 at 20-Jun-2025 21:31:51 CEST
Begin processing the 149th record. run: 1793 subRun: 1 event: 26149 at 20-Jun-2025 21:31:53 CEST
Begin processing the 150th record. run: 1793 subRun: 1 event: 26150 at 20-Jun-2025 21:31:55 CEST
Begin processing the 151st record. run: 1793 subRun: 1 event: 26151 at 20-Jun-2025 21:31:57 CEST
Begin processing the 152nd record. run: 1793 subRun: 1 event: 26152 at 20-Jun-2025 21:31:58 CEST
Begin processing the 153rd record. run: 1793 subRun: 1 event: 26153 at 20-Jun-2025 21:32:05 CEST
Begin processing the 154th record. run: 1793 subRun: 1 event: 26154 at 20-Jun-2025 21:32:08 CEST
Begin processing the 155th record. run: 1793 subRun: 1 event: 26155 at 20-Jun-2025 21:32:10 CEST
Begin processing the 156th record. run: 1793 subRun: 1 event: 26156 at 20-Jun-2025 21:32:12 CEST
Begin processing the 157th record. run: 1793 subRun: 1 event: 26157 at 20-Jun-2025 21:32:13 CEST
Begin processing the 158th record. run: 1793 subRun: 1 event: 26158 at 20-Jun-2025 21:32:14 CEST
Begin processing the 159th record. run: 1793 subRun: 1 event: 26159 at 20-Jun-2025 21:32:15 CEST
Begin processing the 160th record. run: 1793 subRun: 1 event: 26160 at 20-Jun-2025 21:32:16 CEST
Begin processing the 161st record. run: 1793 subRun: 1 event: 26161 at 20-Jun-2025 21:32:18 CEST
Begin processing the 162nd record. run: 1793 subRun: 1 event: 26162 at 20-Jun-2025 21:32:19 CEST
Begin processing the 163rd record. run: 1793 subRun: 1 event: 26163 at 20-Jun-2025 21:32:20 CEST
Begin processing the 164th record. run: 1793 subRun: 1 event: 26164 at 20-Jun-2025 21:32:21 CEST
Begin processing the 165th record. run: 1793 subRun: 1 event: 26165 at 20-Jun-2025 21:32:22 CEST
Begin processing the 166th record. run: 1793 subRun: 1 event: 26166 at 20-Jun-2025 21:32:23 CEST
Begin processing the 167th record. run: 1793 subRun: 1 event: 26167 at 20-Jun-2025 21:32:24 CEST
Begin processing the 168th record. run: 1793 subRun: 1 event: 26168 at 20-Jun-2025 21:32:26 CEST
Begin processing the 169th record. run: 1793 subRun: 1 event: 26169 at 20-Jun-2025 21:32:27 CEST
Begin processing the 170th record. run: 1793 subRun: 1 event: 26170 at 20-Jun-2025 21:32:28 CEST
Begin processing the 171st record. run: 1793 subRun: 1 event: 26171 at 20-Jun-2025 21:32:29 CEST
Begin processing the 172nd record. run: 1793 subRun: 1 event: 26172 at 20-Jun-2025 21:32:30 CEST
Begin processing the 173rd record. run: 1793 subRun: 1 event: 26173 at 20-Jun-2025 21:32:31 CEST
Begin processing the 174th record. run: 1793 subRun: 1 event: 26174 at 20-Jun-2025 21:32:32 CEST
Begin processing the 175th record. run: 1793 subRun: 1 event: 26175 at 20-Jun-2025 21:32:39 CEST
Begin processing the 176th record. run: 1793 subRun: 1 event: 26176 at 20-Jun-2025 21:32:40 CEST
Begin processing the 177th record. run: 1793 subRun: 1 event: 26177 at 20-Jun-2025 21:32:41 CEST
Begin processing the 178th record. run: 1793 subRun: 1 event: 26178 at 20-Jun-2025 21:32:42 CEST
Begin processing the 179th record. run: 1793 subRun: 1 event: 26179 at 20-Jun-2025 21:32:46 CEST
Begin processing the 180th record. run: 1793 subRun: 1 event: 26180 at 20-Jun-2025 21:32:57 CEST
Begin processing the 181st record. run: 1793 subRun: 1 event: 26181 at 20-Jun-2025 21:33:05 CEST
Begin processing the 182nd record. run: 1793 subRun: 1 event: 26182 at 20-Jun-2025 21:33:07 CEST
Begin processing the 183rd record. run: 1793 subRun: 1 event: 26183 at 20-Jun-2025 21:33:08 CEST
Begin processing the 184th record. run: 1793 subRun: 1 event: 26184 at 20-Jun-2025 21:33:09 CEST
Begin processing the 185th record. run: 1793 subRun: 1 event: 26185 at 20-Jun-2025 21:33:10 CEST
Begin processing the 186th record. run: 1793 subRun: 1 event: 26186 at 20-Jun-2025 21:33:11 CEST
Begin processing the 187th record. run: 1793 subRun: 1 event: 26187 at 20-Jun-2025 21:33:12 CEST
Begin processing the 188th record. run: 1793 subRun: 1 event: 26188 at 20-Jun-2025 21:33:18 CEST
Begin processing the 189th record. run: 1793 subRun: 1 event: 26189 at 20-Jun-2025 21:33:19 CEST
Begin processing the 190th record. run: 1793 subRun: 1 event: 26190 at 20-Jun-2025 21:33:20 CEST
Begin processing the 191st record. run: 1793 subRun: 1 event: 26191 at 20-Jun-2025 21:33:22 CEST
Begin processing the 192nd record. run: 1793 subRun: 1 event: 26192 at 20-Jun-2025 21:33:27 CEST
Begin processing the 193rd record. run: 1793 subRun: 1 event: 26193 at 20-Jun-2025 21:33:28 CEST
Begin processing the 194th record. run: 1793 subRun: 1 event: 26194 at 20-Jun-2025 21:33:28 CEST
Begin processing the 195th record. run: 1793 subRun: 1 event: 26195 at 20-Jun-2025 21:33:33 CEST
Begin processing the 196th record. run: 1793 subRun: 1 event: 26196 at 20-Jun-2025 21:33:42 CEST
Begin processing the 197th record. run: 1793 subRun: 1 event: 26197 at 20-Jun-2025 21:33:44 CEST
Begin processing the 198th record. run: 1793 subRun: 1 event: 26198 at 20-Jun-2025 21:33:46 CEST
Begin processing the 199th record. run: 1793 subRun: 1 event: 26199 at 20-Jun-2025 21:33:47 CEST
Begin processing the 200th record. run: 1793 subRun: 1 event: 26200 at 20-Jun-2025 21:33:48 CEST
20-Jun-2025 21:33:50 CEST  Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/bf/84/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_130_20231027T174237Z_gen_g4_detsim_hitreco__20240503T043803Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.254303       1.12026       5.58427      0.897631      0.732125        200    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.0198387     0.0349536     0.0873409     0.0395641     0.0126248       200    
prod:pandorapid:Chi2ParticleID                 0.115871      0.180413      0.353618       0.1788       0.0348584       200    
prod:energyreconue:EnergyReco                  0.0895241     0.356736      0.633982      0.359086      0.0761949       200    
prod:energyreconumu:EnergyReco                0.00376222     0.207606       1.60685      0.181746      0.224264        200    
prod:energyreconc:EnergyReco                  0.00341597     0.0460239     0.295014      0.0333807     0.0484714       200    
[art]:TriggerResults:TriggerResultInserter    1.0791e-05    1.85149e-05   0.000412308   1.4908e-05    2.9182e-05       200    
end_path:ana:PIDASelectionAna                 0.00304479     0.293896        3.411       0.0811929     0.458357        200    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 11032.9 MB
  Peak resident set size usage (VmHWM): 1241.09 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
.:
total 76
-rw-r--r-- 1 dune001 lbno 31831 Jun 20 21:33 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_130_20231027T174237Z_gen_g4_detsim_hitreco__20240503T043803Z_reco2_pida_2025-06-20T_192753Z.root
-rw-r--r-- 1 dune001 lbno 23609 Jun 20 21:33 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1793_130_20231027T174237Z_gen_g4_detsim_hitreco__20240503T043803Z_reco2_reco_2025-06-20T_192753Z.log
-rw-r--r-- 1 dune001 lbno 11811 Jun 20 21:33 jobscript.log
-rw-r--r-- 1 dune001 lbno   183 Jun 20 21:33 justin-processed-pfns.txt
-rw-r--r-- 1 dune001 lbno   134 Jun 20 21:27 all-input-dids.txt
-rw-r--r-- 1 dune001 lbno     0 Jun 20 21:28 debugprod.log
justIN time: 2025-08-15 04:55:50 UTC       justIN version: 01.03.02