[CentOS] Tyan K8SRE troubles with CentOS 4.4 i386

Thu Dec 14 00:36:53 UTC 2006
John R Pierce <pierce at hogranch.com>

Dan Halbert wrote:
> >hey, I thought mixing dimm sizes was verbotten on opterons?
> Thanks for everyone's comments. I consulted with the person who has 
> actually been working on the hardware. He says these motherboards have 
> six slots for DIMM's, not 8, but they are supposedly in GT24 chassis, 
> which come with K8SRE boards. So it's not clear what's going on; I may 
> be misreporting the motherboards. There are mixed sizes per board: 4 
> 1GB and 2 2GB. During testing, we did try removing the 2GB DIMM's, but 
> did not rebalance the memory. We are physically away from the 
> machines, so we'll take a closer look in the morning, and try some 
> other configurations. The x86_64 vs. i386 difference is odd; maybe 
> they treat the memory controllers differently?
> Where would I find official warnings about memory-size mixing? We can 
> bring it up with the vendor. The motherboard manual doesn't have such 
> caveats. Is this oral tradition?

my HP DL series Opteron servers have a caveat that you CAN mix memories 
sizes, as long as all the memory on one CPU is the same.   Also, on the 
boards that have 8 dimms per CPU, if you load more than 4 dimms (2 banks 
of dual channel) on a single CPU's memory banks, you have to slow the 
memory bus down a notch.     Now, I just realized, this is an Opteron 
8xx server, 4 CPU sockets, 4 sets of 8 dimms each, its possible the 
Opteron 2xx series (dual CPU sockets) have different memory rules.   As 
far as I know, these rules are from the CPU chips themselves, as the 
CPUs have the memory controllers integrated.

               +-----+                  +-----+
    cpu0 ======| CPU |--hypertransport--| CPU |=====cpu1
membanks ======|  0  |                  |  1  |=====membanks
               +-----+                  +-----+
                  |hypertransport to IO controllers
               | IO  |
               | bus |

(where the 'IO bus' is the main board chipset that manages all IO busses 
on the system)

anyways...  perusing the manual on the k8sre...  hmmm.   its not 
specified at all, and there's 2 x 2 slots on each processor, which does 
look like the 2xx have a different sort of memory controller than the 8xx