21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.
Jobsub ID 413077.142@justin-prod-sched01.dune.hep.ac.uk
| Jobsub ID | 413077.142@justin-prod-sched01.dune.hep.ac.uk | |
| Workflow ID | 7645 | |
| Stage ID | 1 | |
| User name | hsouza@fnal.gov | |
| HTCondor Group | group_dune | |
| Requested | Processors | 1 |
| GPU | No | |
| RSS bytes | 3145728000 (3000 MiB) | |
| Wall seconds limit | 80000 (22 hours) | |
| Submitted time | 2025-06-14 05:46:40 | |
| Site | UK_Bristol | |
| Entry | CMSHTPC_T2_UK_SGrid_Bristol_lcgce02 | |
| Last heartbeat | 2025-06-14 06:17:35 | |
| From worker node | Hostname | hd89.dice.priv |
| cpuinfo | AMD EPYC 7713 64-Core Processor | |
| OS release | Scientific Linux release 7.9 (Nitrogen) | |
| Processors | 1 | |
| RSS bytes | 3145728000 (3000 MiB) | |
| Wall seconds limit | 259200 (72 hours) | |
| GPU | ||
| Inner Apptainer? | True | |
| Job state | outputting_failed | |
| Allocator name | justin-allocator-pro.dune.hep.ac.uk | |
| Started | 2025-06-14 05:47:50 | |
| Input files | fardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_351_20231122T122019Z_gen_g4_detsim_hitreco__20240503T063829Z_reco2.root | |
| Jobscript | Exit code | 0 |
| Real time | 7m (472s) | |
| CPU time | 0m (56s = 11%) | |
| Max RSS bytes | 1448546304 (1381 MiB) | |
| Outputting started | 2025-06-14 05:55:43 | |
| Output files | ||
| Finished | 2025-06-14 06:17:35 | |
| List job events Wrapper job log | ||
Jobscript log (last 10,000 characters)
6209058 subRun: 1 event: 70330 at 14-Jun-2025 05:52:50 UTC Begin processing the 131st record. run: 6209058 subRun: 1 event: 70331 at 14-Jun-2025 05:52:52 UTC Begin processing the 132nd record. run: 6209058 subRun: 1 event: 70332 at 14-Jun-2025 05:52:54 UTC Begin processing the 133rd record. run: 6209058 subRun: 1 event: 70333 at 14-Jun-2025 05:52:55 UTC Begin processing the 134th record. run: 6209058 subRun: 1 event: 70334 at 14-Jun-2025 05:52:57 UTC Begin processing the 135th record. run: 6209058 subRun: 1 event: 70335 at 14-Jun-2025 05:52:59 UTC Begin processing the 136th record. run: 6209058 subRun: 1 event: 70336 at 14-Jun-2025 05:53:01 UTC Begin processing the 137th record. run: 6209058 subRun: 1 event: 70337 at 14-Jun-2025 05:53:03 UTC Begin processing the 138th record. run: 6209058 subRun: 1 event: 70338 at 14-Jun-2025 05:53:05 UTC Begin processing the 139th record. run: 6209058 subRun: 1 event: 70339 at 14-Jun-2025 05:53:07 UTC Begin processing the 140th record. run: 6209058 subRun: 1 event: 70340 at 14-Jun-2025 05:53:08 UTC Begin processing the 141st record. run: 6209058 subRun: 1 event: 70341 at 14-Jun-2025 05:53:10 UTC Begin processing the 142nd record. run: 6209058 subRun: 1 event: 70342 at 14-Jun-2025 05:53:12 UTC Begin processing the 143rd record. run: 6209058 subRun: 1 event: 70343 at 14-Jun-2025 05:53:13 UTC Begin processing the 144th record. run: 6209058 subRun: 1 event: 70344 at 14-Jun-2025 05:53:14 UTC Begin processing the 145th record. run: 6209058 subRun: 1 event: 70345 at 14-Jun-2025 05:53:21 UTC Begin processing the 146th record. run: 6209058 subRun: 1 event: 70346 at 14-Jun-2025 05:53:23 UTC Begin processing the 147th record. run: 6209058 subRun: 1 event: 70347 at 14-Jun-2025 05:53:26 UTC Begin processing the 148th record. run: 6209058 subRun: 1 event: 70348 at 14-Jun-2025 05:53:27 UTC Begin processing the 149th record. run: 6209058 subRun: 1 event: 70349 at 14-Jun-2025 05:53:29 UTC Begin processing the 150th record. run: 6209058 subRun: 1 event: 70350 at 14-Jun-2025 05:53:31 UTC Begin processing the 151st record. run: 6209058 subRun: 1 event: 70351 at 14-Jun-2025 05:53:33 UTC Begin processing the 152nd record. run: 6209058 subRun: 1 event: 70352 at 14-Jun-2025 05:53:34 UTC Begin processing the 153rd record. run: 6209058 subRun: 1 event: 70353 at 14-Jun-2025 05:53:38 UTC Begin processing the 154th record. run: 6209058 subRun: 1 event: 70354 at 14-Jun-2025 05:53:40 UTC Begin processing the 155th record. run: 6209058 subRun: 1 event: 70355 at 14-Jun-2025 05:53:41 UTC Begin processing the 156th record. run: 6209058 subRun: 1 event: 70356 at 14-Jun-2025 05:53:43 UTC Begin processing the 157th record. run: 6209058 subRun: 1 event: 70357 at 14-Jun-2025 05:53:45 UTC Begin processing the 158th record. run: 6209058 subRun: 1 event: 70358 at 14-Jun-2025 05:53:50 UTC Begin processing the 159th record. run: 6209058 subRun: 1 event: 70359 at 14-Jun-2025 05:53:52 UTC Begin processing the 160th record. run: 6209058 subRun: 1 event: 70360 at 14-Jun-2025 05:53:54 UTC Begin processing the 161st record. run: 6209058 subRun: 1 event: 70361 at 14-Jun-2025 05:53:56 UTC Begin processing the 162nd record. run: 6209058 subRun: 1 event: 70362 at 14-Jun-2025 05:53:58 UTC Begin processing the 163rd record. run: 6209058 subRun: 1 event: 70363 at 14-Jun-2025 05:53:59 UTC Begin processing the 164th record. run: 6209058 subRun: 1 event: 70364 at 14-Jun-2025 05:54:01 UTC Begin processing the 165th record. run: 6209058 subRun: 1 event: 70365 at 14-Jun-2025 05:54:03 UTC Begin processing the 166th record. run: 6209058 subRun: 1 event: 70366 at 14-Jun-2025 05:54:04 UTC Begin processing the 167th record. run: 6209058 subRun: 1 event: 70367 at 14-Jun-2025 05:54:06 UTC Begin processing the 168th record. run: 6209058 subRun: 1 event: 70368 at 14-Jun-2025 05:54:08 UTC Begin processing the 169th record. run: 6209058 subRun: 1 event: 70369 at 14-Jun-2025 05:54:10 UTC Begin processing the 170th record. run: 6209058 subRun: 1 event: 70370 at 14-Jun-2025 05:54:12 UTC Begin processing the 171st record. run: 6209058 subRun: 1 event: 70371 at 14-Jun-2025 05:54:14 UTC Begin processing the 172nd record. run: 6209058 subRun: 1 event: 70372 at 14-Jun-2025 05:54:14 UTC Begin processing the 173rd record. run: 6209058 subRun: 1 event: 70373 at 14-Jun-2025 05:54:16 UTC Begin processing the 174th record. run: 6209058 subRun: 1 event: 70374 at 14-Jun-2025 05:54:25 UTC Begin processing the 175th record. run: 6209058 subRun: 1 event: 70375 at 14-Jun-2025 05:54:27 UTC Begin processing the 176th record. run: 6209058 subRun: 1 event: 70376 at 14-Jun-2025 05:54:29 UTC Begin processing the 177th record. run: 6209058 subRun: 1 event: 70377 at 14-Jun-2025 05:54:30 UTC Begin processing the 178th record. run: 6209058 subRun: 1 event: 70378 at 14-Jun-2025 05:54:32 UTC Begin processing the 179th record. run: 6209058 subRun: 1 event: 70379 at 14-Jun-2025 05:54:47 UTC Begin processing the 180th record. run: 6209058 subRun: 1 event: 70380 at 14-Jun-2025 05:54:49 UTC Begin processing the 181st record. run: 6209058 subRun: 1 event: 70381 at 14-Jun-2025 05:54:50 UTC Begin processing the 182nd record. run: 6209058 subRun: 1 event: 70382 at 14-Jun-2025 05:54:52 UTC Begin processing the 183rd record. run: 6209058 subRun: 1 event: 70383 at 14-Jun-2025 05:54:54 UTC Begin processing the 184th record. run: 6209058 subRun: 1 event: 70384 at 14-Jun-2025 05:54:56 UTC Begin processing the 185th record. run: 6209058 subRun: 1 event: 70385 at 14-Jun-2025 05:54:58 UTC Begin processing the 186th record. run: 6209058 subRun: 1 event: 70386 at 14-Jun-2025 05:55:00 UTC Begin processing the 187th record. run: 6209058 subRun: 1 event: 70387 at 14-Jun-2025 05:55:02 UTC Begin processing the 188th record. run: 6209058 subRun: 1 event: 70388 at 14-Jun-2025 05:55:04 UTC Begin processing the 189th record. run: 6209058 subRun: 1 event: 70389 at 14-Jun-2025 05:55:06 UTC Begin processing the 190th record. run: 6209058 subRun: 1 event: 70390 at 14-Jun-2025 05:55:07 UTC Begin processing the 191st record. run: 6209058 subRun: 1 event: 70391 at 14-Jun-2025 05:55:09 UTC Begin processing the 192nd record. run: 6209058 subRun: 1 event: 70392 at 14-Jun-2025 05:55:11 UTC Begin processing the 193rd record. run: 6209058 subRun: 1 event: 70393 at 14-Jun-2025 05:55:13 UTC Begin processing the 194th record. run: 6209058 subRun: 1 event: 70394 at 14-Jun-2025 05:55:15 UTC Begin processing the 195th record. run: 6209058 subRun: 1 event: 70395 at 14-Jun-2025 05:55:17 UTC Begin processing the 196th record. run: 6209058 subRun: 1 event: 70396 at 14-Jun-2025 05:55:19 UTC Begin processing the 197th record. run: 6209058 subRun: 1 event: 70397 at 14-Jun-2025 05:55:21 UTC Begin processing the 198th record. run: 6209058 subRun: 1 event: 70398 at 14-Jun-2025 05:55:22 UTC Begin processing the 199th record. run: 6209058 subRun: 1 event: 70399 at 14-Jun-2025 05:55:24 UTC Begin processing the 200th record. run: 6209058 subRun: 1 event: 70400 at 14-Jun-2025 05:55:26 UTC 14-Jun-2025 05:55:28 UTC Closed input file "root://se1.farm.particle.cz:1094//dune/RSE/fardet-hd/a3/9e/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_351_20231122T122019Z_gen_g4_detsim_hitreco__20240503T063829Z_reco2.root" ================================================================================================================================ TimeTracker printout (sec) Min Avg Max Median RMS nEvts ================================================================================================================================ Full event 0.427998 1.79759 14.1941 1.53579 1.54024 200 -------------------------------------------------------------------------------------------------------------------------------- source:RootInput(read) 0.0241015 0.230991 12.7371 0.0484333 1.30982 200 prod:pandorapid:Chi2ParticleID 0.14208 0.219716 0.335866 0.21881 0.03026 200 prod:energyreconue:EnergyReco 0.17528 0.512402 1.0147 0.501863 0.13854 200 prod:energyreconumu:EnergyReco 0.00242987 0.174376 6.8956 0.148266 0.48484 200 prod:energyreconc:EnergyReco 0.00224697 0.0368272 0.504076 0.03472 0.0496395 200 [art]:TriggerResults:TriggerResultInserter 8.927e-06 1.3479e-05 4.2661e-05 1.1121e-05 5.0394e-06 200 end_path:ana:PIDASelectionAna 0.0746254 0.622814 2.75933 0.567167 0.293009 200 ================================================================================================================================ ==================================================================================================== MemoryTracker summary (base-10 MB units used) Peak virtual memory usage (VmPeak) : 37015.8 MB Peak resident set size usage (VmHWM): 1448.55 MB ==================================================================================================== Art has completed and will exit with status 0. === End last 100 lines of lar log file === .: total 159 -rw-r--r-- 1 dnpil001 users 124908 Jun 14 05:55 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_351_20231122T122019Z_gen_g4_detsim_hitreco__20240503T063829Z_reco2_pida_2025-06-14T_054756Z.root -rw-r--r-- 1 dnpil001 users 24015 Jun 14 05:55 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_6209058_351_20231122T122019Z_gen_g4_detsim_hitreco__20240503T063829Z_reco2_reco_2025-06-14T_054756Z.log -rw-r--r-- 1 dnpil001 users 11970 Jun 14 05:55 jobscript.log -rw-r--r-- 1 dnpil001 users 186 Jun 14 05:55 justin-processed-pfns.txt -rw-r--r-- 1 dnpil001 users 137 Jun 14 05:47 all-input-dids.txt -rw-r--r-- 1 dnpil001 users 0 Jun 14 05:48 debugprod.log