Jobsub ID 166774.18@justin-prod-sched02.dune.hep.ac.uk

Jobsub ID	166774.18@justin-prod-sched02.dune.hep.ac.uk
Workflow ID	5755
Stage ID	1
User name	walkerj@fnal.gov
HTCondor Group	group_dune
Requested	Processors	1
	GPU	No
	RSS bytes	13631488000 (13000 MiB)
	Wall seconds limit	80000 (22 hours)
Submitted time	2025-03-27 15:54:37
Site	US_FNAL-FermiGrid
Entry	FNAL_GPGrid_ce03_mcore_op_duneonly
Last heartbeat	2025-03-27 16:06:53
From worker node	Hostname	dunegli-4304842-0-fnpc9065.fnal.gov
	cpuinfo	Intel(R) Xeon(R) CPU E5-2680 v4 @ 2.40GHz
	OS release	Scientific Linux release 7.9 (Nitrogen)
	Processors	1
	RSS bytes	13631488000 (13000 MiB)
	Wall seconds limit	172800 (48 hours)
	GPU
	Inner Apptainer?	True
Job state	finished
Allocator name	justin-allocator-pro.dune.hep.ac.uk
Started	2025-03-27 16:02:55
Input files	usertests:np04_raw_run011655_0002_dl11_reco1_2025-03-27T_122025Z.root
Jobscript	Exit code	0
	Real time	1m (112s)
	CPU time	1m (105s = 93%)
	Max RSS bytes	4606652416 (4393 MiB)
Outputting started	2025-03-27 16:04:48
Output files	usertests:np04_raw_run011655_0002_dl11_reco1_2025-03-27T_122025Z_reco2_2025-03-27T_160300Z.root
Finished	2025-03-27 16:06:53
Saved logs	justin-logs:166774.18-justin-prod-sched02.dune.hep.ac.uk.logs.tgz
List job events Wrapper job log

Jobscript log (last 10,000 characters)

Input PFN = root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/b0/f5/np04_raw_run011655_0002_dl11_reco1_2025-03-27T_122025Z.root
Setting up larsoft UPS area... /cvmfs/larsoft.opensciencegrid.org
Setting up DUNE UPS area... /cvmfs/dune.opensciencegrid.org/products/dune/

MRB_PROJECT=larsoft
MRB_PROJECT_VERSION=v09_82_00
MRB_QUALS=e26:prof
MRB_TOP=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/db94055707bcdcf9f1ecafa647855eda4a6bbe26/my_v09_82_00
MRB_SOURCE=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/db94055707bcdcf9f1ecafa647855eda4a6bbe26/my_v09_82_00/srcs
MRB_BUILDDIR=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/db94055707bcdcf9f1ecafa647855eda4a6bbe26/my_v09_82_00/build_slf7.x86_64
MRB_INSTALL=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/db94055707bcdcf9f1ecafa647855eda4a6bbe26/my_v09_82_00/localProducts_larsoft_v09_82_00_e26_prof

PRODUCTS=/cvmfs/fifeuser3.opensciencegrid.org/sw/dune/db94055707bcdcf9f1ecafa647855eda4a6bbe26/my_v09_82_00/localProducts_larsoft_v09_82_00_e26_prof:/cvmfs/dune.opensciencegrid.org/products/dune:/cvmfs/larsoft.opensciencegrid.org/products:/cvmfs/larsoft.opensciencegrid.org/packages:/cvmfs/fermilab.opensciencegrid.org/products/common/db/
CETPKG_INSTALL=/exp/dune/app/users/walkerj/my_v09_82_00/localProducts_larsoft_v09_82_00_e26_prof

local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/db94055707bcdcf9f1ecafa647855eda4a6bbe26/my_v09_82_00/localProducts_larsoft_v09_82_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
local product directory is /cvmfs/fifeuser3.opensciencegrid.org/sw/dune/db94055707bcdcf9f1ecafa647855eda4a6bbe26/my_v09_82_00/localProducts_larsoft_v09_82_00_e26_prof
----------- this block should be empty ------------------
---------------------------------------------------------
/cvmfs/larsoft.opensciencegrid.org/products/xrootd/v5_4_3b/Linux64bit+3.10-2.17-e26-p3913-prof/lib/libXrdPosixPreload.so
=== Start last 100 lines of lar log file ===
77 0
78 0
79 0
80 0
81 0
82 0
83 0
84 0
85 0
86 0
87 0
88 0
89 0
90 0
91 0
92 0
93 0
94 0
95 0
96 0
97 0
98 0
99 0
Begin processing the 26th record. run: 11655 subRun: 1 event: 646 at 27-Mar-2025 16:04:46 UTC
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:0 345 XUs and 488 XVs -> 0 XUVs
C:0 T:1 169 XUs and 204 XVs -> 0 XUVs
C:0 T:2 432 XUs and 1362 XVs -> 2 XUVs
C:0 T:5 45 XUs and 39 XVs -> 0 XUVs
C:0 T:6 614 XUs and 269 XVs -> 2 XUVs
C:0 T:9 180 XUs and 360 XVs -> 0 XUVs
C:0 T:10 6 XUs and 2 XVs -> 0 XUVs
4 XUVs total
3 collection wire objects
4 potential space points
Neighbour search...
8 tests to find 4 neighbours
Iterating with no regularization...
Begin: 416.935
0 404.37
1 404.37
Now with regularization...
Begin: 347.053
0 347.052
Begin processing the 27th record. run: 11655 subRun: 1 event: 656 at 27-Mar-2025 16:04:46 UTC
72 X, 28 U, 34 V bad channels
Finding XUV coincidences...
C:0 T:1 232 XUs and 172 XVs -> 0 XUVs
C:0 T:2 696 XUs and 1138 XVs -> 1 XUVs
C:0 T:5 120 XUs and 50 XVs -> 0 XUVs
C:0 T:6 595 XUs and 2694 XVs -> 0 XUVs
C:0 T:9 526 XUs and 161 XVs -> 0 XUVs
C:0 T:10 199 XUs and 0 XVs -> 0 XUVs
C:0 T:11 8 XUs and 2 XVs -> 0 XUVs
1 XUVs total
1 collection wire objects
1 potential space points
Neighbour search...
1 tests to find 0 neighbours
Iterating with no regularization...
Begin: 184.202
0 184.202
Now with regularization...
Begin: 152.669
0 152.669
27-Mar-2025 16:04:47 UTC  Closed output file "np04_raw_run011655_0002_dl11_reco1_2025-03-27T_122025Z_reco2_2025-03-27T_160300Z.root"
27-Mar-2025 16:04:47 UTC  Closed input file "root://fndca1.fnal.gov:1094/pnfs/fnal.gov/usr/dune/persistent/staging/usertests/b0/f5/np04_raw_run011655_0002_dl11_reco1_2025-03-27T_122025Z.root"

================================================================================================================================
TimeTracker printout (sec)                        Min           Avg           Max         Median          RMS         nEvts   
================================================================================================================================
Full event                                    0.00991735     0.0248058     0.0855973     0.0221159     0.0133426       27     
--------------------------------------------------------------------------------------------------------------------------------
source:RootInput(read)                        0.000527632   0.00210269     0.0361999    0.000727175   0.00669171       27     
reco:reco3d:SpacePointSolver                  0.00433018     0.0116544     0.0285194     0.0102391    0.00487071       27     
reco:hitpdune:DisambigFromSpacePoints         0.00200623    0.00425829     0.0117943    0.00392708    0.00172573       27     
[art]:TriggerResults:TriggerResultInserter    1.6109e-05    1.92182e-05   5.7426e-05    1.7492e-05    7.66612e-06      27     
end_path:out1:RootOutput                       2.677e-06    3.89989e-06   1.9156e-05     3.295e-06    3.01651e-06      27     
end_path:out1:RootOutput(write)               0.00257804    0.00633736     0.0109418    0.00605434    0.00195453       27     
================================================================================================================================

====================================================================================================
MemoryTracker summary (base-10 MB units used)

  Peak virtual memory usage (VmPeak)  : 9307.03 MB
  Peak resident set size usage (VmHWM): 4606.65 MB
  Details saved in: 'mem.db'
====================================================================================================
ToolBasedRawDigitPrepService:dtor: Event count: 0
ToolBasedRawDigitPrepService:dtor:  Call count: 0
ToolBasedRawDigitPrepService:dtor: Time report for 7 tools.
ToolBasedRawDigitPrepService:dtor:                    digitReader:   0.00 sec
ToolBasedRawDigitPrepService:dtor:                adcSampleFiller:   0.00 sec
ToolBasedRawDigitPrepService:dtor:                adcScaleAdcToKe:   0.00 sec
ToolBasedRawDigitPrepService:dtor:          pdspTailPedRemovalZKe:   0.00 sec
ToolBasedRawDigitPrepService:dtor:         adcKeepAllSignalFinder:   0.00 sec
ToolBasedRawDigitPrepService:dtor:                adcScaleKeToAdc:   0.00 sec
ToolBasedRawDigitPrepService:dtor:         pdsp_RemoveBadChannels:   0.00 sec
Art has completed and will exit with status 0.
=== End last 100 lines of lar log file ===
lar exit code 0