--On Thursday, June 30, 2011 05:04:07 AM +0800 Emmanuel Noobadmin centos.admin@gmail.com wrote:
On 6/30/11, Les Mikesell lesmikesell@gmail.com wrote:
The seriously on-the-cheap approach is to run a few virtual servers on hardware slightly better than one of the individual servers would need.
Actually THAT is the fundamental problem ;) The physical server is frankly much more powerful than the two guest running on it. I have the same applications + public web/email running on old dual core machines with less memory than the guests.
I don't recall you mentioning which VM solution you're using.
Some problematic areas that I've seen when using VMs:
+ memory ballooning sometimes causes problems (I've not actually seen it, but I've seen various warnings about having it enabled and resultant flakiness, and I run with it disabled)
+ I/O stacks not doing TCP segment offload correctly. This is an ugly one that bit me hard and took a while to track down. It's happened in both ESXi and Xen (and I'm not saying that KVM isn't affected, either).
The symptoms of this is things seem to be fine under low load, but as network traffic starts to increase TCP sessions start stalling out or dying. I've seen it to the point where I can't even maintain an ssh session long enough to get a login prompt.
What it comes down to is the top level (virtual) OS decides to hand off TCP segmentation to the (virtual) NIC. To make a long story short, between the guest OS, the virtual NICs, the virtual switches, the host OS, and the physical NICs, there exists a path (depending on versions and hardware) where everyone things somebody else is doing TCP segment handling, and nobody is. So as I/O goes up or fragmentation occurs, the protocol goes into the toilet. Sometimes you miss packets and sometimes the data is corrupt.
Disabling tcp segment offload in both the host and guest avoids the problem (forcing the OS to do it instead of the VM & physical layers). Be aware of reboots and update processes that want to reenable it ...
Devin