John R Pierce wrote:

Jim Wildman wrote:

...

'over commit' of RAM is the phrase you are probably looking for. And this is one of the limitations of Xen (it doesn't).

perhaps I am being naive here, but I'd guess that a hypervisor which allowed such would have to have its own page swapping mechanism, as it needs to present each guest OS with the appearance of having whatever "physical" memory it thinks it has... meanwhile, each guest OS has *its* own page swapping to implement whatever VM scheme it supports.

since most modern virtual memory systems attempt to maximize their memory footprint by treating the bulk of otherwise unused ('free') pages as disk cache, I can't help but think this wouldn't work that well (having the hypervisor doing swapping on pages that are guest OS discardable cache pages just doesn't sound very effective)

VMware's technology is twofold:

- Transparent page sharing - every so often the hypervisor scans for identical memory pages and eliminates them, if your running a lot of similar VMs you can regain quite a bit of memory, there is a minimal amount of overhead and you can control how often the page scan occurs(or disable it entirely)

An extreme example: http://blogs.vmware.com/virtualreality/2008/03/memory-overcomm.html

[..]a screenshot of their environment showing a total of 178 VMs running on the system. You can also see in the screenshot that less than 20 GB of RAM out of the total 64 GB of RAM is being used on the system. With a total of 178 VMs configured for 512 MB of RAM each they are currently allocating 89 GB of memory to running VMs which means they are oversubscribed on the host.

- Memory ballooning - when the VM host is starved for physical memory and the vmware balloon driver is loaded into the guest VMs, the hypervisor tells that driver to consume more memory, forcing the guest VMs to free up things like buffer cache and if it has to go to swap. This process does not occur if you have plenty of physical memory available on the host.

Combine both of these with Vmware DRS and a cluster and you can potentially overcommit a ton of memory depending on your workload.

The hypervisor can swap as well, as there is memory overhead above and beyond what the guest OS thinks it is using. But of course any active swapping kills performance.

I think some bleeding edge KVM versions or patches offer these features, I have never used KVM or Xen myself. Also haven't used the enterprise versions of vmware, Foundation is good enough for me for now.

nate

Larry wrote:

Thank you, for all your answers!

What about the other question? Can clustering help in completing this task? I thought cluster nodes are sharing RAM on default ( obviously I was wrong).

Depends what your goal is, if you want to run a cluster with shared memory between systems, you typically need a very high speed interconnect and very low latencies, your not likely to achieve this within VMs.

Your original message just talked about sharing ram between VMs, you didn't mention why you were specifically interested in doing that.

nate, which vmware you use? VMware ESXi or VMWare Server or something else?

Whenever possible I'd use ESXi, I happen to use vmware server on the system that is sending my email because it's old and ESXi doesn't support the 3ware 8006-2 SATA card in it. Check the hardware compatibility list, the list is fairly short for some things like SATA, though ESX 4 is due out in the coming week(s) which is supposed to dramatically improve SATA support.

Note that out of the box ESXi(with the free license) doesn't support snapshots or cold migrations(between servers) at least last time I checked the feature comparisons sheet. I'll be installing about 12 new ESXi systems soon. VMware server does not support page sharing or memory ballooning, and does support snapshots. I use vmware server 1.x, probably will go to 2.x in the next couple months.

p.s. Stable xen, has ballooning. You can say for every VM: memory=1024MB, max-mem=2048MB. And you can tweak the memory "manually", without having to restart the VM and without any other problems.

That's not memory ballooning, at least not in the vmware world, and you can do the same in vmware, though if you decide to drop the amount of physical memory allocated to the guest without having the guest free up that memory you'll go into swap pretty quick, so it's not a practice I do, really ever.

If Xen has a specific balloon driver like vmware and inflates the balloon when you change the memory settings that would be pretty cool, as far as I know vmware does not do that, the balloon only kicks in when host memory is low, not when you decide you want to resize the guest.

nate