justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 231327.38@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID231327.38@justin-prod-sched02.dune.hep.ac.uk
Workflow ID7983
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-06-27 12:54:25
SiteUS_UChicago
EntryEngage_US_MWT2_uct2_gk02_condce_mcore
Last heartbeat2025-06-27 14:14:01
From worker nodeHostnameuct2-c644.mwt2.org
cpuinfoAMD EPYC 7302 16-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-27 13:54:44
Input filesfardet-hd:nutau_dune10kt_1x2x6_1436_783_20230828T114657Z_gen_g4_detsim_hitreco__20240220T133419Z_reco2.root
JobscriptExit code0
Real time19m (1141s)
CPU time18m (1088s = 95%)
Max RSS bytes1504337920 (1434 MiB)
Outputting started2025-06-27 14:13:47
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nutau_0/07983/1/001/HierarchyRecoPerformance_nutau_dune10kt_1x2x6_1436_783_20230828T114657Z_gen_g4_detsim_hitreco__20240220T133419Z_reco2.root
Finished2025-06-27 14:14:01
Saved logsjustin-logs:231327.38-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v10_08_00
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19
MRB_SOURCE=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/srcs
MRB_BUILDDIR=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/localProducts_larsoft_v10_08_00_e26_prof

PRODUCTS=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/localProducts_larsoft_v10_08_00_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/localProducts_larsoft_v10_08_00_e26_prof

local product directory is /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/localProducts_larsoft_v10_08_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/4b/59/nutau_dune10kt_1x2x6_1436_783_20230828T114657Z_gen_g4_detsim_hitreco__20240220T133419Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
YES RECO: 2212 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 1000010020 GEN: 3
YES RECO: 2212 GEN: 4
YES RECO: 2212 GEN: 3
YES RECO: 2212 GEN: 3
YES RECO: -211 GEN: 3
Begin processing the 67th record. run: 1436 subRun: 1 event: 78367 at 27-Jun-2025 09:11:37 CDT
Begin processing the 68th record. run: 1436 subRun: 1 event: 78368 at 27-Jun-2025 09:11:39 CDT
YES RECO: -11 GEN: 3
Begin processing the 69th record. run: 1436 subRun: 1 event: 78369 at 27-Jun-2025 09:11:41 CDT
Begin processing the 70th record. run: 1436 subRun: 1 event: 78370 at 27-Jun-2025 09:11:43 CDT
Begin processing the 71st record. run: 1436 subRun: 1 event: 78371 at 27-Jun-2025 09:11:46 CDT
Begin processing the 72nd record. run: 1436 subRun: 1 event: 78372 at 27-Jun-2025 09:11:48 CDT
Begin processing the 73rd record. run: 1436 subRun: 1 event: 78373 at 27-Jun-2025 09:11:50 CDT
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
Begin processing the 74th record. run: 1436 subRun: 1 event: 78374 at 27-Jun-2025 09:11:54 CDT
YES RECO: 22 GEN: 3
Begin processing the 75th record. run: 1436 subRun: 1 event: 78375 at 27-Jun-2025 09:11:56 CDT
Begin processing the 76th record. run: 1436 subRun: 1 event: 78376 at 27-Jun-2025 09:11:59 CDT
Begin processing the 77th record. run: 1436 subRun: 1 event: 78377 at 27-Jun-2025 09:12:01 CDT
Begin processing the 78th record. run: 1436 subRun: 1 event: 78378 at 27-Jun-2025 09:12:05 CDT
Begin processing the 79th record. run: 1436 subRun: 1 event: 78379 at 27-Jun-2025 09:12:08 CDT
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
Begin processing the 80th record. run: 1436 subRun: 1 event: 78380 at 27-Jun-2025 09:12:12 CDT
Begin processing the 81st record. run: 1436 subRun: 1 event: 78381 at 27-Jun-2025 09:12:14 CDT
YES RECO: 2212 GEN: 3
YES RECO: 2212 GEN: 3
YES RECO: 211 GEN: 3
YES RECO: -11 GEN: 4
YES RECO: 2212 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 211 GEN: 3
YES RECO: -11 GEN: 3
Begin processing the 82nd record. run: 1436 subRun: 1 event: 78382 at 27-Jun-2025 09:12:17 CDT
YES RECO: 2212 GEN: 3
YES RECO: 2212 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: -211 GEN: 3
Begin processing the 83rd record. run: 1436 subRun: 1 event: 78383 at 27-Jun-2025 09:12:30 CDT
YES RECO: 2212 GEN: 3
Begin processing the 84th record. run: 1436 subRun: 1 event: 78384 at 27-Jun-2025 09:12:32 CDT
Begin processing the 85th record. run: 1436 subRun: 1 event: 78385 at 27-Jun-2025 09:12:35 CDT
YES RECO: 2212 GEN: 3
Begin processing the 86th record. run: 1436 subRun: 1 event: 78386 at 27-Jun-2025 09:12:37 CDT
YES RECO: 211 GEN: 3
YES RECO: 211 GEN: 4
YES RECO: -11 GEN: 5
Begin processing the 87th record. run: 1436 subRun: 1 event: 78387 at 27-Jun-2025 09:12:40 CDT
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: -211 GEN: 3
Begin processing the 88th record. run: 1436 subRun: 1 event: 78388 at 27-Jun-2025 09:12:43 CDT
Begin processing the 89th record. run: 1436 subRun: 1 event: 78389 at 27-Jun-2025 09:12:45 CDT
YES RECO: -211 GEN: 3
Begin processing the 90th record. run: 1436 subRun: 1 event: 78390 at 27-Jun-2025 09:12:48 CDT
Begin processing the 91st record. run: 1436 subRun: 1 event: 78391 at 27-Jun-2025 09:12:50 CDT
Begin processing the 92nd record. run: 1436 subRun: 1 event: 78392 at 27-Jun-2025 09:12:52 CDT
Begin processing the 93rd record. run: 1436 subRun: 1 event: 78393 at 27-Jun-2025 09:13:12 CDT
Begin processing the 94th record. run: 1436 subRun: 1 event: 78394 at 27-Jun-2025 09:13:14 CDT
Begin processing the 95th record. run: 1436 subRun: 1 event: 78395 at 27-Jun-2025 09:13:16 CDT
Begin processing the 96th record. run: 1436 subRun: 1 event: 78396 at 27-Jun-2025 09:13:18 CDT
Begin processing the 97th record. run: 1436 subRun: 1 event: 78397 at 27-Jun-2025 09:13:21 CDT
Begin processing the 98th record. run: 1436 subRun: 1 event: 78398 at 27-Jun-2025 09:13:23 CDT
YES RECO: -11 GEN: 3
Begin processing the 99th record. run: 1436 subRun: 1 event: 78399 at 27-Jun-2025 09:13:25 CDT
Begin processing the 100th record. run: 1436 subRun: 1 event: 78400 at 27-Jun-2025 09:13:42 CDT
27-Jun-2025 09:13:45 CDT  Closed output file "nutau_dune10kt_1x2x6_1436_783_20230828T114657Z_gen_g4_detsim_hitreco__20240220T133419Z_reco2_reco2.root"
27-Jun-2025 09:13:45 CDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/4b/59/nutau_dune10kt_1x2x6_1436_783_20230828T114657Z_gen_g4_detsim_hitreco__20240220T133419Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                     0.0355388      10.5254       464.404       2.21148       51.0154        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.00484801     0.0295165      1.06828      0.0177949     0.104755        100    
reco:pandora:StandardPandora                   0.0224178      10.4846       464.277       2.18885       51.0044        100    
[art]:TriggerResults:TriggerResultInserter     9.989e-06    1.92971e-05   8.7715e-05    1.63755e-05   1.07946e-05      100    
end_path:out1:RootOutput                       2.625e-06    4.96776e-06   2.8063e-05     3.927e-06    2.80999e-06      100    
end_path:out1:RootOutput(write)               0.000886571    0.0110409     0.102144     0.00430948     0.0158669       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2494.61 MB
  Peak resident set size usage (VmHWM): 1504.34 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2025-08-14 21:52:44 UTC       justIN version: 01.03.02