NUMA support not enabled in amd64/config.server

[Peter Cordes]

In the Jaunty Linux git tree

grep NUMA debian/config/*/config*
debian/config/amd64/config:# CONFIG_NUMA is not set

 Most multi-socket K8/K10 systems are NUMA. (The ones that aren't have memory attached to only one of the CPUs. I guess even that's sort of NUMA, because memory access is slower for the cores in the second socket, but they don't have any fast memory to allocate from.)

 Upcoming multi-socket Nehalem machines will be NUMA, also, because Intel is following AMD's lead of putting the memory controller on the CPU. So memory bandwidth can scale with number of sockets, and all that.

From an Intel presentation at IDF about optimizing for Nehalem (aka Core i7), which I found a copy of here:
http://www.benchmark.rs/tests/editorial/Nehalem_munich/presentations/Software-Tuning_for_Nehalem.pdf

  Expect most Multi-socket Intel® CoreTM i7 processor platforms to use NUMA
• Locality matters
   – Remote memory access latency ~1.7x than local memory
   – local memory bandwidth can be up to 2x greater than remote

 So that's an even bigger non-uniformity than K8. I think K8 can manage full bandwidth to non-local memory, it's just latency that takes a hit. I think K10 is similar.

 During Jaunty's lifecycle, _all_ new latest-generation 2P and larger servers will be NUMA machines. Even if it has some overhead on non-NUMA machines, it would be a very good idea to compile in support for NUMA, to get NUMA-aware memory allocation and process scheduling.

 CONFIG_NUMA is marked as experimental for i386, but the built-in help for the option recommends it for amd64.

 I would not recommend CONFIG_NUMA for the -generic flavour, just -server. Multi-socket desktops are rare. As long as voluntary pre-emption isn't a big deal, people with NUMA desktops should run the -server flavour. If they do want voluntary pre-emption and NUMA, they should compile their own.

currently (2009/Feb), though, anyone who wants Linux NUMA support has to compile their own kernel.

[Bryan Wu (cooloney)]

We will get some idea from server team and make the decision soon.

-Bryan

[Bryan Wu (cooloney)]

After discussion in UDS, we enabled the NUMA for amd64 server config. So change to Fix-Committed of this bug.