justIN           Dashboard       Workflows       Jobs       AWT       Sites       Storages       Docs       Login

21 July 2025: This instance at RAL is read-only. Please do not try submitting new workflows for now.

Jobsub ID 231327.22@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID231327.22@justin-prod-sched02.dune.hep.ac.uk
Workflow ID7983
Stage ID1
User nameimawby@fnal.gov
HTCondor Groupgroup_dune
RequestedProcessors1
GPUNo
RSS bytes4194304000 (4000 MiB)
Wall seconds limit43200 (12 hours)
Submitted time2025-06-27 12:54:25
SiteUS_UChicago
EntryEngage_US_MWT2_uiuc_gk02_condce_mcore
Last heartbeat2025-06-27 14:13:15
From worker nodeHostnamemwt2-c037.campuscluster.illinois.edu
cpuinfoAMD EPYC 7443 24-Core Processor
OS releaseScientific Linux release 7.9 (Nitrogen)
Processors1
RSS bytes4194304000 (4000 MiB)
Wall seconds limit86400 (24 hours)
GPU
Inner Apptainer?True
Job statefinished
Allocator namejustin-allocator-pro.dune.hep.ac.uk
Started2025-06-27 13:55:16
Input filesfardet-hd:nutau_dune10kt_1x2x6_1107_453_20230826T173405Z_gen_g4_detsim_hitreco__20240219T201815Z_reco2.root
JobscriptExit code0
Real time17m (1052s)
CPU time15m (934s = 88%)
Max RSS bytes1417768960 (1352 MiB)
Outputting started2025-06-27 14:12:49
Output fileshttps://fndcadoor.fnal.gov:2880/dune/scratch/users/imawby/splitting_nutau_0/07983/1/001/HierarchyRecoPerformance_nutau_dune10kt_1x2x6_1107_453_20230826T173405Z_gen_g4_detsim_hitreco__20240219T201815Z_reco2.root
Finished2025-06-27 14:13:15
Saved logsjustin-logs:231327.22-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events     Wrapper job log

Jobscript log (last 10,000 characters)

Justin processors: 1
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v10_08_00
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19
MRB_SOURCE=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/srcs
MRB_BUILDDIR=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/localProducts_larsoft_v10_08_00_e26_prof

PRODUCTS=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/localProducts_larsoft_v10_08_00_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/localProducts_larsoft_v10_08_00_e26_prof

local product directory is /cvmfs/fifeuser1.opensciencegrid.org/sw/dune/ba960f2d78c1c5ee2ebb9823e8434f1c8c704e19/localProducts_larsoft_v10_08_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/af/ba/nutau_dune10kt_1x2x6_1107_453_20230826T173405Z_gen_g4_detsim_hitreco__20240219T201815Z_reco2.root
lar exit code 0
=== Start last 100 lines of lar log file ===
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 2212 GEN: 3
YES RECO: 2212 GEN: 3
YES RECO: 211 GEN: 3
YES RECO: 211 GEN: 3
YES RECO: 2212 GEN: 4
Begin processing the 62nd record. run: 1107 subRun: 1 event: 45362 at 27-Jun-2025 09:09:49 CDT
Begin processing the 63rd record. run: 1107 subRun: 1 event: 45363 at 27-Jun-2025 09:09:52 CDT
Begin processing the 64th record. run: 1107 subRun: 1 event: 45364 at 27-Jun-2025 09:09:54 CDT
Begin processing the 65th record. run: 1107 subRun: 1 event: 45365 at 27-Jun-2025 09:09:57 CDT
YES RECO: -11 GEN: 3
Begin processing the 66th record. run: 1107 subRun: 1 event: 45366 at 27-Jun-2025 09:10:00 CDT
Begin processing the 67th record. run: 1107 subRun: 1 event: 45367 at 27-Jun-2025 09:10:03 CDT
YES RECO: 2212 GEN: 3
YES RECO: -211 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
Begin processing the 68th record. run: 1107 subRun: 1 event: 45368 at 27-Jun-2025 09:10:08 CDT
YES RECO: 2212 GEN: 3
Begin processing the 69th record. run: 1107 subRun: 1 event: 45369 at 27-Jun-2025 09:10:11 CDT
YES RECO: 2212 GEN: 3
Begin processing the 70th record. run: 1107 subRun: 1 event: 45370 at 27-Jun-2025 09:10:41 CDT
Begin processing the 71st record. run: 1107 subRun: 1 event: 45371 at 27-Jun-2025 09:10:44 CDT
Begin processing the 72nd record. run: 1107 subRun: 1 event: 45372 at 27-Jun-2025 09:10:47 CDT
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
Begin processing the 73rd record. run: 1107 subRun: 1 event: 45373 at 27-Jun-2025 09:10:50 CDT
Begin processing the 74th record. run: 1107 subRun: 1 event: 45374 at 27-Jun-2025 09:10:53 CDT
Begin processing the 75th record. run: 1107 subRun: 1 event: 45375 at 27-Jun-2025 09:10:55 CDT
YES RECO: 2212 GEN: 3
YES RECO: -211 GEN: 3
YES RECO: 2212 GEN: 3
YES RECO: 211 GEN: 3
Begin processing the 76th record. run: 1107 subRun: 1 event: 45376 at 27-Jun-2025 09:10:57 CDT
Begin processing the 77th record. run: 1107 subRun: 1 event: 45377 at 27-Jun-2025 09:11:00 CDT
Begin processing the 78th record. run: 1107 subRun: 1 event: 45378 at 27-Jun-2025 09:11:02 CDT
Begin processing the 79th record. run: 1107 subRun: 1 event: 45379 at 27-Jun-2025 09:11:05 CDT
YES RECO: 211 GEN: 3
YES RECO: -11 GEN: 4
Begin processing the 80th record. run: 1107 subRun: 1 event: 45380 at 27-Jun-2025 09:11:08 CDT
Begin processing the 81st record. run: 1107 subRun: 1 event: 45381 at 27-Jun-2025 09:11:11 CDT
YES RECO: 2212 GEN: 3
Begin processing the 82nd record. run: 1107 subRun: 1 event: 45382 at 27-Jun-2025 09:11:15 CDT
YES RECO: 211 GEN: 3
YES RECO: -211 GEN: 3
YES RECO: -11 GEN: 3
YES RECO: -11 GEN: 3
Begin processing the 83rd record. run: 1107 subRun: 1 event: 45383 at 27-Jun-2025 09:11:21 CDT
Begin processing the 84th record. run: 1107 subRun: 1 event: 45384 at 27-Jun-2025 09:11:24 CDT
Begin processing the 85th record. run: 1107 subRun: 1 event: 45385 at 27-Jun-2025 09:11:27 CDT
Begin processing the 86th record. run: 1107 subRun: 1 event: 45386 at 27-Jun-2025 09:11:29 CDT
Begin processing the 87th record. run: 1107 subRun: 1 event: 45387 at 27-Jun-2025 09:11:31 CDT
Begin processing the 88th record. run: 1107 subRun: 1 event: 45388 at 27-Jun-2025 09:11:34 CDT
YES RECO: 2212 GEN: 3
YES RECO: 2212 GEN: 3
Begin processing the 89th record. run: 1107 subRun: 1 event: 45389 at 27-Jun-2025 09:11:38 CDT
Begin processing the 90th record. run: 1107 subRun: 1 event: 45390 at 27-Jun-2025 09:11:41 CDT
Begin processing the 91st record. run: 1107 subRun: 1 event: 45391 at 27-Jun-2025 09:11:44 CDT
Begin processing the 92nd record. run: 1107 subRun: 1 event: 45392 at 27-Jun-2025 09:11:47 CDT
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
Begin processing the 93rd record. run: 1107 subRun: 1 event: 45393 at 27-Jun-2025 09:11:49 CDT
Begin processing the 94th record. run: 1107 subRun: 1 event: 45394 at 27-Jun-2025 09:11:52 CDT
Begin processing the 95th record. run: 1107 subRun: 1 event: 45395 at 27-Jun-2025 09:12:01 CDT
Begin processing the 96th record. run: 1107 subRun: 1 event: 45396 at 27-Jun-2025 09:12:07 CDT
Begin processing the 97th record. run: 1107 subRun: 1 event: 45397 at 27-Jun-2025 09:12:12 CDT
Begin processing the 98th record. run: 1107 subRun: 1 event: 45398 at 27-Jun-2025 09:12:15 CDT
Begin processing the 99th record. run: 1107 subRun: 1 event: 45399 at 27-Jun-2025 09:12:18 CDT
Begin processing the 100th record. run: 1107 subRun: 1 event: 45400 at 27-Jun-2025 09:12:21 CDT
YES RECO: 2212 GEN: 3
YES RECO: -211 GEN: 3
YES RECO: 211 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: -211 GEN: 3
YES RECO: 22 GEN: 3
YES RECO: 22 GEN: 3
27-Jun-2025 09:12:47 CDT  Closed output file "nutau_dune10kt_1x2x6_1107_453_20230826T173405Z_gen_g4_detsim_hitreco__20240219T201815Z_reco2_reco2.root"
27-Jun-2025 09:12:47 CDT  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/fardet-hd/af/ba/nutau_dune10kt_1x2x6_1107_453_20230826T173405Z_gen_g4_detsim_hitreco__20240219T201815Z_reco2.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                      1.90124       8.78129       243.029       2.81247       25.8051        100    
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                         0.015238      0.0439982     0.524286      0.0370289     0.0544502       100    
reco:pandora:StandardPandora                    1.8771        8.72054       242.872       2.73934       25.7887        100    
[art]:TriggerResults:TriggerResultInserter    1.2835e-05    3.33595e-05   0.000112071   2.5964e-05    2.15075e-05      100    
end_path:out1:RootOutput                       2.555e-06    6.86499e-06   3.4174e-05     6.422e-06    4.34336e-06      100    
end_path:out1:RootOutput(write)               0.00122763     0.0164443     0.130133      0.0054591     0.0239628       100    
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 2428.26 MB
  Peak resident set size usage (VmHWM): 1417.77 MB
====================================================================================================
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
justIN time: 2025-08-14 21:53:06 UTC       justIN version: 01.03.02