[CentOS] NUMA split mode?

Sun Oct 1 16:39:40 UTC 2017
John R Pierce <pierce at hogranch.com>

On 10/1/2017 8:38 AM, hw wrote:
> HP says that what they call "NUMA split mode" should be disabled in the
> BIOS of the Z800 workstation when running Linux.  They are reasoning
> that Linux kernels do not support this feature and even might not boot
> if it´s enabled.

hmm, that workstation is a dual Xeon 56xx (Westmere-EP, derived from 
Nehalem), new in 2010

> Since it apparently was years ago since they made this statement, I´m
> wondering if I should still leave this feature disabled or not.  More
> recent kernels might support it, and it´s supposed to improve
> performance.
> Could someone explain what this feature actually is or does, and if
> Centos kernels support it?

On these sorts of dual socket hardware architectures, half of the memory 
is directly attached to each CPU, and the two CPUs are linked with a QPI 
bus.  All the memory appears in one unified address space, but the 
memory belonging to the 'other' CPU has a little higher latency to 
access since it has to go across the QPI.   In non-NUMA mode, this is 
ignored, and all memory is treated as equal from the OS perspective.  in 
NUMA mode, an attempt is made to keep process memory on one CPU's 
memory, and to prefer scheduling those processes on the cores of that 
CPU. This can get messy, say you have a process running on core 0 (in 
cpu0) which allocates a big block of shared memory, then spawns 8 worker 
threads which all run concurrently and use this same shared working 
memory space.   there's only 4 or 6 cores on each of the two CPUs, so 
either these worker threads have to wait for an available core on the 
same CPU as the memory allocation, or some of them end up running across 
the QPI bus anyways.

I believe Linux, even RHEL 6, does support NUMA configurations, but its 
very questionable if a random typical workload would actually gain much 
from it, and it adds significant overhead in keeping track of all this.

john r pierce, recycling bits in santa cruz