justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 229369.75@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID229369.75@justin-prod-sched02.dune.hep.ac.uk
Workflow ID7926
Stage ID1
User namehsouza@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes3145728000 (3000 MiB)
Wall seconds limit80000 (22 hours)
Submitted time2025-06-25 15:57:21
SiteUK_Manchester
EntryUBoone_T2_UK_Manchester_ce01
Last heartbeat2025-06-25 16:33:50
From worker nodeHostnamewn1207081.tier2.hep.manchester.ac.uk
cpuinfoAMD EPYC 7702P 64-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes3145728000 (3000 MiB)
Wall seconds limit257400 (71 hours)
GPU
Inner Apptainer?True
Job statestalled
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-25 16:00:31
Input filesfardet-hd:atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1497_386_20231027T144707Z_gen_g4_detsim_hitreco__20240503T040206Z_reco2.root
Outputting started2025-06-25 16:07:20
Output files
Finished2025-06-25 17:13:54
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

processing the 128th record. run: 1497 subRun: 1 event: 77328 at 25-Jun-2025 17:04:49 BST
Begin processing the 129th record. run: 1497 subRun: 1 event: 77329 at 25-Jun-2025 17:04:50 BST
Begin processing the 130th record. run: 1497 subRun: 1 event: 77330 at 25-Jun-2025 17:04:51 BST
Begin processing the 131st record. run: 1497 subRun: 1 event: 77331 at 25-Jun-2025 17:04:53 BST
Begin processing the 132nd record. run: 1497 subRun: 1 event: 77332 at 25-Jun-2025 17:04:55 BST
Begin processing the 133rd record. run: 1497 subRun: 1 event: 77333 at 25-Jun-2025 17:04:57 BST
Begin processing the 134th record. run: 1497 subRun: 1 event: 77334 at 25-Jun-2025 17:04:59 BST
Begin processing the 135th record. run: 1497 subRun: 1 event: 77335 at 25-Jun-2025 17:05:00 BST
Begin processing the 136th record. run: 1497 subRun: 1 event: 77336 at 25-Jun-2025 17:05:02 BST
Begin processing the 137th record. run: 1497 subRun: 1 event: 77337 at 25-Jun-2025 17:05:04 BST
Begin processing the 138th record. run: 1497 subRun: 1 event: 77338 at 25-Jun-2025 17:05:07 BST
Begin processing the 139th record. run: 1497 subRun: 1 event: 77339 at 25-Jun-2025 17:05:09 BST
Begin processing the 140th record. run: 1497 subRun: 1 event: 77340 at 25-Jun-2025 17:05:11 BST
Begin processing the 141st record. run: 1497 subRun: 1 event: 77341 at 25-Jun-2025 17:05:13 BST
Begin processing the 142nd record. run: 1497 subRun: 1 event: 77342 at 25-Jun-2025 17:05:15 BST
Begin processing the 143rd record. run: 1497 subRun: 1 event: 77343 at 25-Jun-2025 17:05:17 BST
Begin processing the 144th record. run: 1497 subRun: 1 event: 77344 at 25-Jun-2025 17:05:18 BST
Begin processing the 145th record. run: 1497 subRun: 1 event: 77345 at 25-Jun-2025 17:05:20 BST
Begin processing the 146th record. run: 1497 subRun: 1 event: 77346 at 25-Jun-2025 17:05:21 BST
Begin processing the 147th record. run: 1497 subRun: 1 event: 77347 at 25-Jun-2025 17:05:23 BST
Begin processing the 148th record. run: 1497 subRun: 1 event: 77348 at 25-Jun-2025 17:05:25 BST
Begin processing the 149th record. run: 1497 subRun: 1 event: 77349 at 25-Jun-2025 17:05:26 BST
Begin processing the 150th record. run: 1497 subRun: 1 event: 77350 at 25-Jun-2025 17:05:30 BST
Begin processing the 151st record. run: 1497 subRun: 1 event: 77351 at 25-Jun-2025 17:05:32 BST
Begin processing the 152nd record. run: 1497 subRun: 1 event: 77352 at 25-Jun-2025 17:05:34 BST
Begin processing the 153rd record. run: 1497 subRun: 1 event: 77353 at 25-Jun-2025 17:05:36 BST
Begin processing the 154th record. run: 1497 subRun: 1 event: 77354 at 25-Jun-2025 17:05:38 BST
Begin processing the 155th record. run: 1497 subRun: 1 event: 77355 at 25-Jun-2025 17:05:39 BST
Begin processing the 156th record. run: 1497 subRun: 1 event: 77356 at 25-Jun-2025 17:05:40 BST
Begin processing the 157th record. run: 1497 subRun: 1 event: 77357 at 25-Jun-2025 17:05:42 BST
Begin processing the 158th record. run: 1497 subRun: 1 event: 77358 at 25-Jun-2025 17:05:46 BST
Begin processing the 159th record. run: 1497 subRun: 1 event: 77359 at 25-Jun-2025 17:05:53 BST
Begin processing the 160th record. run: 1497 subRun: 1 event: 77360 at 25-Jun-2025 17:05:55 BST
Begin processing the 161st record. run: 1497 subRun: 1 event: 77361 at 25-Jun-2025 17:05:56 BST
Begin processing the 162nd record. run: 1497 subRun: 1 event: 77362 at 25-Jun-2025 17:05:58 BST
Begin processing the 163rd record. run: 1497 subRun: 1 event: 77363 at 25-Jun-2025 17:06:00 BST
Begin processing the 164th record. run: 1497 subRun: 1 event: 77364 at 25-Jun-2025 17:06:02 BST
Begin processing the 165th record. run: 1497 subRun: 1 event: 77365 at 25-Jun-2025 17:06:03 BST
Begin processing the 166th record. run: 1497 subRun: 1 event: 77366 at 25-Jun-2025 17:06:05 BST
Begin processing the 167th record. run: 1497 subRun: 1 event: 77367 at 25-Jun-2025 17:06:07 BST
Begin processing the 168th record. run: 1497 subRun: 1 event: 77368 at 25-Jun-2025 17:06:09 BST
Begin processing the 169th record. run: 1497 subRun: 1 event: 77369 at 25-Jun-2025 17:06:12 BST
Begin processing the 170th record. run: 1497 subRun: 1 event: 77370 at 25-Jun-2025 17:06:14 BST
Begin processing the 171st record. run: 1497 subRun: 1 event: 77371 at 25-Jun-2025 17:06:15 BST
Begin processing the 172nd record. run: 1497 subRun: 1 event: 77372 at 25-Jun-2025 17:06:20 BST
Begin processing the 173rd record. run: 1497 subRun: 1 event: 77373 at 25-Jun-2025 17:06:22 BST
Begin processing the 174th record. run: 1497 subRun: 1 event: 77374 at 25-Jun-2025 17:06:24 BST
Begin processing the 175th record. run: 1497 subRun: 1 event: 77375 at 25-Jun-2025 17:06:26 BST
Begin processing the 176th record. run: 1497 subRun: 1 event: 77376 at 25-Jun-2025 17:06:28 BST
Begin processing the 177th record. run: 1497 subRun: 1 event: 77377 at 25-Jun-2025 17:06:29 BST
Begin processing the 178th record. run: 1497 subRun: 1 event: 77378 at 25-Jun-2025 17:06:32 BST
Begin processing the 179th record. run: 1497 subRun: 1 event: 77379 at 25-Jun-2025 17:06:34 BST
Begin processing the 180th record. run: 1497 subRun: 1 event: 77380 at 25-Jun-2025 17:06:36 BST
Begin processing the 181st record. run: 1497 subRun: 1 event: 77381 at 25-Jun-2025 17:06:37 BST
Begin processing the 182nd record. run: 1497 subRun: 1 event: 77382 at 25-Jun-2025 17:06:39 BST
Begin processing the 183rd record. run: 1497 subRun: 1 event: 77383 at 25-Jun-2025 17:06:41 BST
Begin processing the 184th record. run: 1497 subRun: 1 event: 77384 at 25-Jun-2025 17:06:43 BST
Begin processing the 185th record. run: 1497 subRun: 1 event: 77385 at 25-Jun-2025 17:06:44 BST
Begin processing the 186th record. run: 1497 subRun: 1 event: 77386 at 25-Jun-2025 17:06:46 BST
Begin processing the 187th record. run: 1497 subRun: 1 event: 77387 at 25-Jun-2025 17:06:48 BST
Begin processing the 188th record. run: 1497 subRun: 1 event: 77388 at 25-Jun-2025 17:06:50 BST
Begin processing the 189th record. run: 1497 subRun: 1 event: 77389 at 25-Jun-2025 17:06:53 BST
Begin processing the 190th record. run: 1497 subRun: 1 event: 77390 at 25-Jun-2025 17:06:55 BST
Begin processing the 191st record. run: 1497 subRun: 1 event: 77391 at 25-Jun-2025 17:06:56 BST
Begin processing the 192nd record. run: 1497 subRun: 1 event: 77392 at 25-Jun-2025 17:06:58 BST
Begin processing the 193rd record. run: 1497 subRun: 1 event: 77393 at 25-Jun-2025 17:07:00 BST
Begin processing the 194th record. run: 1497 subRun: 1 event: 77394 at 25-Jun-2025 17:07:02 BST
Begin processing the 195th record. run: 1497 subRun: 1 event: 77395 at 25-Jun-2025 17:07:03 BST
Begin processing the 196th record. run: 1497 subRun: 1 event: 77396 at 25-Jun-2025 17:07:06 BST
Begin processing the 197th record. run: 1497 subRun: 1 event: 77397 at 25-Jun-2025 17:07:08 BST
Begin processing the 198th record. run: 1497 subRun: 1 event: 77398 at 25-Jun-2025 17:07:10 BST
Begin processing the 199th record. run: 1497 subRun: 1 event: 77399 at 25-Jun-2025 17:07:16 BST
Begin processing the 200th record. run: 1497 subRun: 1 event: 77400 at 25-Jun-2025 17:07:18 BST
25-Jun-2025 17:07:19 BST  Closed input file "root://xrootd.echo.stfc.ac.uk:1094/dune:/protodune/RSE/fardet-hd/18/31/atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1497_386_20231027T144707Z_gen_g4_detsim_hitreco__20240503T040206Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.424832       1.33928       4.85585       1.28062      0.485756        200    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00903471     0.0207769     0.182614      0.0181739     0.0240174       200    
prod:pandorapid:Chi2ParticleID                 0.0523557     0.0823577     0.119563      0.0848268     0.0116406       200    
prod:energyreconue:EnergyReco                  0.234067      0.530627       1.11573      0.506893      0.153661        200    
prod:energyreconumu:EnergyReco                0.00114214     0.130101       1.80693      0.0966872     0.189393        200    
prod:energyreconc:EnergyReco                   0.0024579     0.0701815     0.395489      0.0746707     0.0671443       200    
[art]:TriggerResults:TriggerResultInserter     9.468e-06    1.38429e-05   7.5141e-05     1.31e-05     4.99488e-06      200    
end_path:ana:PIDASelectionAna                  0.0489335     0.504724       2.04834      0.480769      0.217254        200    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 19796 MB
  Peak resident set size usage (VmHWM): 1421.9 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
.:
total 80
-rw-r--r-- 1 dune01 dune01 28753 Jun 25 17:07 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1497_386_20231027T144707Z_gen_g4_detsim_hitreco__20240503T040206Z_reco2_pida_2025-06-25T_160035Z.root
-rw-r--r-- 1 dune01 dune01 23439 Jun 25 17:07 atmnu_max_weighted_randompolicy_dune10kt_1x2x6_1497_386_20231027T144707Z_gen_g4_detsim_hitreco__20240503T040206Z_reco2_reco_2025-06-25T_160035Z.log
-rw-r--r-- 1 dune01 dune01 11754 Jun 25 17:07 jobscript.log
-rw-r--r-- 1 dune01 dune01   195 Jun 25 17:07 justin-processed-pfns.txt
-rw-r--r-- 1 dune01 dune01   134 Jun 25 17:00 all-input-dids.txt
-rw-r--r-- 1 dune01 dune01     0 Jun 25 17:00 debugprod.log
justIN time: 2025-08-14 21:48:27 UTC       justIN version: 01.03.02