On Mon, Mar 28, 2011 at 08:59:09AM -0400, Steve Thompson wrote: > On Mon, 28 Mar 2011, Pasi Kärkkäinen wrote: > >> On Sun, Mar 27, 2011 at 09:41:04AM -0400, Steve Thompson wrote: >>> First. With Xen I was never able to start more than 30 guests at one time >>> with any success; the 31st guest always failed to boot or crashed during >>> booting, no matter which guest I chose as the 31st. With KVM I chose to >>> add more guests to see if it could be done, with the result that I now >>> have 36 guests running simultaneously. >> >> Hmm.. I think I've seen that earlier. I *think* it was some trivial >> thing to fix, like increasing number of available loop devices or so. > > I tried that, and other things, but was never able to make it work. I was > using max_loop=64 in the end, but even with a larger number I couldn't > start more than 30 guests. Number 31 would fail to boot, and would boot > successfully if I shut down, say, #17. Then #17 would fail to boot, and > so on. > Ok. I have a link somewhere about how to fix that, but I can't seem to be able to find it now. >> Hmm.. Windows 7 might be too new for Xen 3.1 in el5, so for win7 >> upgrading to xen 3.4 or 4.x helps. (gitco.de has newer xen rpms for el5 >> if you're ok with thirdparty rpms). > > Point taken; I realize this. > >>> Third. I was never able to successfully complete a PXE-based installation >>> under Xen. No problems with KVM. >> That's weird. I do that often. What was the problem? > > I use the DHCP server (on the host) to supply all address and name > information, and this works without any issues. In the PXE case, I was > never able to get the guest to communicate with the server for long > enough to fully load pxelinux.0, in spite of the bridge setup. I have no > idea why; it's not exactly rocket science either. > Ok. >> Can you post more info about the benchmark? How many vcpus did the VMs have? >> How much memory? Were the VMs 32b or 64b ? > > The benchmark is just a "make" of a large package of my own > implementation. A top-level makefile drives a series of makes of a set of > sub-packages, 33 of them. It is a compilation of about 1100 C and C++ > source files, including generation of dependencies and binaries, and > running a set of perl scripts (some of which generate some of the C > source). All of the sources and target directories were NFS volumes; only > the local O/S disks were virtualized. I used 1 vcpu per guest and either > 512MB or 1GB of memory. The results I showed were for 64-bit guests with > 512MB memory, but they were qualitatively the same for 32-bit guests. > Increasing memory from 512MB to 1GB made no significant difference to the > timings. Some areas of the build are serial by nature; the result of > 14:38 for KVM w/virtio was changed to 9:52 with vcpu=2 and make -j2. > So it's a fork-heavy workload.. I'll try to make some benchmarks/comparisons soon aswell, also with other workloads. > The 64-bit HVM guests w/o PV were quite a bit faster than the 32-bit HVM > guests, as expected. I also had some Fedora diskless guests (no PV) using > an NFS root, in which situation the 32-bit guests were faster than the > 64-bit guests (and both were faster than the HVM guests w/o PV). These > used kernels that I built myself. > > I did not compare Xen vs KVM with vcpu > 1. > Ok. >> Did you try Xen HVM with PV drivers? > > Yes, but I don't have the exact timings to hand anymore. They were faster > than the non-PV case but still slower than KVM w/virtio. > Yep. >>> Fifth: I love being able to run top/iostat/etc on the host and see just >>> what the hardware is really up to, and to be able to overcommit memory. >> "xm top" and iostat in dom0 works well for me :) > > I personally don't care much for "xm top", and it doesn't help anyway if > you're not running as root or have sudo access, or if you'd like to read > performance info for the whole shebang via /proc (as I do). > -- Pasi