21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 220559.13@justin-prod-sched02.dune.hep.ac.uk
| Jobsub ID | 220559.13@justin-prod-sched02.dune.hep.ac.uk | |
| Workflow ID | 7645 | |
| Stage ID | 1 | |
| User name | hsouza@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 3145728000 (3000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-14 03:15:44 | |
| Site | UK_Bristol | |
| Entry | CMSHTPC_T2_UK_SGrid_Bristol_lcgce02 | |
| Last heartbeat | 2025-06-14 06:17:18 | |
| From worker node | Hostname | hd88.dice.priv |
| cpuinfo | AMD EPYC 7713 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 3145728000 (3000 MiB) | |
| Wall seconds limit | 259200 (72 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-14 05:41:32 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_350_20231121T150121Z_gen_g4_detsim_hitreco__20240507T181058Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 5m (355s) | |
| CPU time | 0m (41s = 11%) | |
| Max RSS bytes | 1031573504 (983 MiB) | |
| Outputting started | 2025-06-14 05:47:27 | |
| Output files | ||
| Finished | 2025-06-14 06:17:18 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
ent: 70130 at 14-Jun-2025 05:45:31 UTC Begin processing the 131st record. run: 6210996 subRun: 1 event: 70131 at 14-Jun-2025 05:45:33 UTC Begin processing the 132nd record. run: 6210996 subRun: 1 event: 70132 at 14-Jun-2025 05:45:35 UTC Begin processing the 133rd record. run: 6210996 subRun: 1 event: 70133 at 14-Jun-2025 05:45:37 UTC Begin processing the 134th record. run: 6210996 subRun: 1 event: 70134 at 14-Jun-2025 05:45:39 UTC Begin processing the 135th record. run: 6210996 subRun: 1 event: 70135 at 14-Jun-2025 05:45:41 UTC Begin processing the 136th record. run: 6210996 subRun: 1 event: 70136 at 14-Jun-2025 05:45:43 UTC Begin processing the 137th record. run: 6210996 subRun: 1 event: 70137 at 14-Jun-2025 05:45:45 UTC Begin processing the 138th record. run: 6210996 subRun: 1 event: 70138 at 14-Jun-2025 05:45:45 UTC Begin processing the 139th record. run: 6210996 subRun: 1 event: 70139 at 14-Jun-2025 05:45:47 UTC Begin processing the 140th record. run: 6210996 subRun: 1 event: 70140 at 14-Jun-2025 05:45:48 UTC Begin processing the 141st record. run: 6210996 subRun: 1 event: 70141 at 14-Jun-2025 05:45:52 UTC Begin processing the 142nd record. run: 6210996 subRun: 1 event: 70142 at 14-Jun-2025 05:45:54 UTC Begin processing the 143rd record. run: 6210996 subRun: 1 event: 70143 at 14-Jun-2025 05:45:55 UTC Begin processing the 144th record. run: 6210996 subRun: 1 event: 70144 at 14-Jun-2025 05:45:57 UTC Begin processing the 145th record. run: 6210996 subRun: 1 event: 70145 at 14-Jun-2025 05:45:59 UTC Begin processing the 146th record. run: 6210996 subRun: 1 event: 70146 at 14-Jun-2025 05:45:59 UTC Begin processing the 147th record. run: 6210996 subRun: 1 event: 70147 at 14-Jun-2025 05:46:01 UTC Begin processing the 148th record. run: 6210996 subRun: 1 event: 70148 at 14-Jun-2025 05:46:02 UTC Begin processing the 149th record. run: 6210996 subRun: 1 event: 70149 at 14-Jun-2025 05:46:04 UTC Begin processing the 150th record. run: 6210996 subRun: 1 event: 70150 at 14-Jun-2025 05:46:05 UTC Begin processing the 151st record. run: 6210996 subRun: 1 event: 70151 at 14-Jun-2025 05:46:07 UTC Begin processing the 152nd record. run: 6210996 subRun: 1 event: 70152 at 14-Jun-2025 05:46:09 UTC Begin processing the 153rd record. run: 6210996 subRun: 1 event: 70153 at 14-Jun-2025 05:46:10 UTC Begin processing the 154th record. run: 6210996 subRun: 1 event: 70154 at 14-Jun-2025 05:46:14 UTC Begin processing the 155th record. run: 6210996 subRun: 1 event: 70155 at 14-Jun-2025 05:46:16 UTC Begin processing the 156th record. run: 6210996 subRun: 1 event: 70156 at 14-Jun-2025 05:46:17 UTC Begin processing the 157th record. run: 6210996 subRun: 1 event: 70157 at 14-Jun-2025 05:46:20 UTC Begin processing the 158th record. run: 6210996 subRun: 1 event: 70158 at 14-Jun-2025 05:46:21 UTC Begin processing the 159th record. run: 6210996 subRun: 1 event: 70159 at 14-Jun-2025 05:46:23 UTC Begin processing the 160th record. run: 6210996 subRun: 1 event: 70160 at 14-Jun-2025 05:46:25 UTC Begin processing the 161st record. run: 6210996 subRun: 1 event: 70161 at 14-Jun-2025 05:46:26 UTC Begin processing the 162nd record. run: 6210996 subRun: 1 event: 70162 at 14-Jun-2025 05:46:28 UTC Begin processing the 163rd record. run: 6210996 subRun: 1 event: 70163 at 14-Jun-2025 05:46:29 UTC Begin processing the 164th record. run: 6210996 subRun: 1 event: 70164 at 14-Jun-2025 05:46:30 UTC Begin processing the 165th record. run: 6210996 subRun: 1 event: 70165 at 14-Jun-2025 05:46:32 UTC Begin processing the 166th record. run: 6210996 subRun: 1 event: 70166 at 14-Jun-2025 05:46:34 UTC Begin processing the 167th record. run: 6210996 subRun: 1 event: 70167 at 14-Jun-2025 05:46:35 UTC Begin processing the 168th record. run: 6210996 subRun: 1 event: 70168 at 14-Jun-2025 05:46:37 UTC Begin processing the 169th record. run: 6210996 subRun: 1 event: 70169 at 14-Jun-2025 05:46:38 UTC Begin processing the 170th record. run: 6210996 subRun: 1 event: 70170 at 14-Jun-2025 05:46:40 UTC Begin processing the 171st record. run: 6210996 subRun: 1 event: 70171 at 14-Jun-2025 05:46:40 UTC Begin processing the 172nd record. run: 6210996 subRun: 1 event: 70172 at 14-Jun-2025 05:46:42 UTC Begin processing the 173rd record. run: 6210996 subRun: 1 event: 70173 at 14-Jun-2025 05:46:43 UTC Begin processing the 174th record. run: 6210996 subRun: 1 event: 70174 at 14-Jun-2025 05:46:45 UTC Begin processing the 175th record. run: 6210996 subRun: 1 event: 70175 at 14-Jun-2025 05:46:46 UTC Begin processing the 176th record. run: 6210996 subRun: 1 event: 70176 at 14-Jun-2025 05:46:48 UTC Begin processing the 177th record. run: 6210996 subRun: 1 event: 70177 at 14-Jun-2025 05:46:50 UTC Begin processing the 178th record. run: 6210996 subRun: 1 event: 70178 at 14-Jun-2025 05:46:52 UTC Begin processing the 179th record. run: 6210996 subRun: 1 event: 70179 at 14-Jun-2025 05:46:53 UTC Begin processing the 180th record. run: 6210996 subRun: 1 event: 70180 at 14-Jun-2025 05:46:55 UTC Begin processing the 181st record. run: 6210996 subRun: 1 event: 70181 at 14-Jun-2025 05:46:56 UTC Begin processing the 182nd record. run: 6210996 subRun: 1 event: 70182 at 14-Jun-2025 05:46:57 UTC Begin processing the 183rd record. run: 6210996 subRun: 1 event: 70183 at 14-Jun-2025 05:46:58 UTC Begin processing the 184th record. run: 6210996 subRun: 1 event: 70184 at 14-Jun-2025 05:47:00 UTC Begin processing the 185th record. run: 6210996 subRun: 1 event: 70185 at 14-Jun-2025 05:47:02 UTC Begin processing the 186th record. run: 6210996 subRun: 1 event: 70186 at 14-Jun-2025 05:47:02 UTC Begin processing the 187th record. run: 6210996 subRun: 1 event: 70187 at 14-Jun-2025 05:47:03 UTC Begin processing the 188th record. run: 6210996 subRun: 1 event: 70188 at 14-Jun-2025 05:47:05 UTC Begin processing the 189th record. run: 6210996 subRun: 1 event: 70189 at 14-Jun-2025 05:47:06 UTC Begin processing the 190th record. run: 6210996 subRun: 1 event: 70190 at 14-Jun-2025 05:47:08 UTC Begin processing the 191st record. run: 6210996 subRun: 1 event: 70191 at 14-Jun-2025 05:47:10 UTC Begin processing the 192nd record. run: 6210996 subRun: 1 event: 70192 at 14-Jun-2025 05:47:11 UTC Begin processing the 193rd record. run: 6210996 subRun: 1 event: 70193 at 14-Jun-2025 05:47:13 UTC Begin processing the 194th record. run: 6210996 subRun: 1 event: 70194 at 14-Jun-2025 05:47:15 UTC Begin processing the 195th record. run: 6210996 subRun: 1 event: 70195 at 14-Jun-2025 05:47:16 UTC Begin processing the 196th record. run: 6210996 subRun: 1 event: 70196 at 14-Jun-2025 05:47:18 UTC Begin processing the 197th record. run: 6210996 subRun: 1 event: 70197 at 14-Jun-2025 05:47:19 UTC Begin processing the 198th record. run: 6210996 subRun: 1 event: 70198 at 14-Jun-2025 05:47:21 UTC Begin processing the 199th record. run: 6210996 subRun: 1 event: 70199 at 14-Jun-2025 05:47:22 UTC Begin processing the 200th record. run: 6210996 subRun: 1 event: 70200 at 14-Jun-2025 05:47:24 UTC 14-Jun-2025 05:47:26 UTC Closed input file "root://ccxrootdegee.in2p3.fr:1094/pnfs/in2p3.fr/data/dune/disk/fardet-hd/2c/7b/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_350_20231121T150121Z_gen_g4_detsim_hitreco__20240507T181058Z_reco2.root" ================================================================================================================================ TimeTracker printout (sec) Min Avg Max Median RMS nEvts ================================================================================================================================ Full event 0.386372 1.38191 17.3282 1.30353 1.16785 200 -------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.0205679 0.128526 16.0532 0.0414336 1.1306 200 prod:pandorapid:Chi2ParticleID 0.121667 0.178658 0.212879 0.18476 0.0256032 200 prod:energyreconue:EnergyReco 0.150962 0.427174 0.719728 0.407443 0.10452 200 prod:energyreconumu:EnergyReco 0.00194519 0.110901 0.58375 0.124566 0.0839192 200 prod:energyreconc:EnergyReco 0.00192075 0.0292019 0.206604 0.0303799 0.0271288 200 [art]:TriggerResults:TriggerResultInserter 7.975e-06 1.55789e-05 6.1537e-05 1.6121e-05 6.81606e-06 200 end_path:ana:PIDASelectionAna 0.0638007 0.506932 1.61613 0.492154 0.155986 200 ================================================================================================================================ ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 36606.2 MB Peak resident set size usage (VmHWM): 1031.57 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === .: total 153 -rw-r--r-- 1 dnpil001 users 118655 Jun 14 05:47 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_350_20231121T150121Z_gen_g4_detsim_hitreco__20240507T181058Z_reco2_pida_2025-06-14T_054138Z.root -rw-r--r-- 1 dnpil001 users 24075 Jun 14 05:47 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6210996_350_20231121T150121Z_gen_g4_detsim_hitreco__20240507T181058Z_reco2_reco_2025-06-14T_054138Z.log -rw-r--r-- 1 dnpil001 users 12010 Jun 14 05:47 jobscript.log -rw-r--r-- 1 dnpil001 users 206 Jun 14 05:47 justin-processed-pfns.txt -rw-r--r-- 1 dnpil001 users 137 Jun 14 05:41 all-input-dids.txt -rw-r--r-- 1 dnpil001 users 0 Jun 14 05:41 debugprod.log