Dan Halbert wrote:
hey, I thought mixing dimm sizes was verbotten on opterons?
Thanks for everyone's comments. I consulted with the person who has actually been working on the hardware. He says these motherboards have six slots for DIMM's, not 8, but they are supposedly in GT24 chassis, which come with K8SRE boards. So it's not clear what's going on; I may be misreporting the motherboards. There are mixed sizes per board: 4 1GB and 2 2GB. During testing, we did try removing the 2GB DIMM's, but did not rebalance the memory. We are physically away from the machines, so we'll take a closer look in the morning, and try some other configurations. The x86_64 vs. i386 difference is odd; maybe they treat the memory controllers differently?
Where would I find official warnings about memory-size mixing? We can bring it up with the vendor. The motherboard manual doesn't have such caveats. Is this oral tradition?
my HP DL series Opteron servers have a caveat that you CAN mix memories sizes, as long as all the memory on one CPU is the same. Also, on the boards that have 8 dimms per CPU, if you load more than 4 dimms (2 banks of dual channel) on a single CPU's memory banks, you have to slow the memory bus down a notch. Now, I just realized, this is an Opteron 8xx server, 4 CPU sockets, 4 sets of 8 dimms each, its possible the Opteron 2xx series (dual CPU sockets) have different memory rules. As far as I know, these rules are from the CPU chips themselves, as the CPUs have the memory controllers integrated.
+-----+ +-----+ cpu0 ======| CPU |--hypertransport--| CPU |=====cpu1 membanks ======| 0 | | 1 |=====membanks +-----+ +-----+ | |hypertransport to IO controllers | +-----+ | IO | | bus | +-----+
(where the 'IO bus' is the main board chipset that manages all IO busses on the system)
anyways... perusing the manual on the k8sre... hmmm. its not specified at all, and there's 2 x 2 slots on each processor, which does look like the 2xx have a different sort of memory controller than the 8xx