Also just to review, clocksource=acpi_pm should be used in conjunction with the tools.synctime = "true" flag in your vmx file. The combination of the two settings prevents time from going into the future from too many ticks, and synctime corrects slow clocks, which leads to a much, much better clock sync.

We'll have to wait until someone figures out a clever way to tie VM clock ticks to a multiplexed physical clock source; until then, clocksync will always be a problem without a complete solution (read up on it). This is as close as it gets!

Allen Tsang wrote:

clocksource=pit is confirmed not working in VMware ESX. You should be using clocksource=acpi_pm in addition to divider=10 to reduce idle load.

Binding to a single CPU is hardly a fix. Always engineer *real* solutions, not poor workarounds! ;)

Kai Schaetzl wrote:

...

Akemi Yagi wrote on Sat, 3 May 2008 06:06:31 -0700:

...

I tried it but it seems to have the same problem as before - when used with clocksource=pit, it hangs on bootup.

For the record, this can also happen in other situations with VMWare. For instance, I have seen that happen with a Suse 9.0 guest on VMWare Server that is running on Win2k3. I was trying clocksource=pit because the clock was jumping ahead of time like nothing. I figured that it is actually a problem with the Suse kernel not liking that specific option (it didn't hang with other clock options). I fixed the time problem by binding the virtual machine to one CPU core. I didn't even have to shut off the power saving features of the CPU.

Kai

---

CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt

(looks at the name of the list "centos-virt").

Well, if you're using the standard rhel centos 5.1 kernel, yes, it should work. and if you can't use tools.... well, you shouldn't even try running your OS in a production virtualized environment because well that's just silly, isn't it?

Speaking of which, I was talking to some friends about building a fully paravirtualized rhel/centos that works with xen, vmware, virtualbox, etc. Do you guys feel like that's a product you would consider using?

- Allen

Kai Schaetzl wrote:

Allen Tsang wrote on Sat, 3 May 2008 15:57:57 -0400:

...

Also just to review, clocksource=acpi_pm should be used in conjunction with the tools.synctime = "true" flag in your vmx file.

And you have verified that that kernel I was talking of even knows "clocksource=acpi_pm" and that I am able to run VMWare Tools? ;-)

Kai

Hey folks,

By Paravirtualization, I mean the installation of "tools" or "guest additions" type packages, which present virtual interfaces to the guest OS. So in VMware, a component of this would mean setting "ethernet0.virtualDev = vmxnet", and having the tools modules pre-installed. A fully virtualized OS for VMware would support all that crap.

From Wikipedia: """ In computing, paravirtualization is a virtualization technique that presents a software interface to virtual machines that is similar but not identical to that of the underlying hardware.

Paravirtualization may allow the virtual machine monitor (VMM) to be simpler or virtual machines that run on it to achieve performance closer to non-virtualized hardware. However, operating systems must be explicitly ported to run on top of a paravirtualized VMM. Owners of proprietary operating systems may decline to allow paravirtualization for strategic purposes. """

I'm neurotic about paravirtualization because, well, we roll our organization's production systems on VMs, so I want minimal performance hit if possible across the board. I'm sure some of you are thinking right now how foolish I am for doing this. In reality, cost savings in labor through all of the flexibilities of a fully virtualized environment is more important to us than raw performance and low latency.

One problem with setting your VM template to utilize virtualDevices in a production environment currently is the inability to PXE boot the machine to provision your host, because no commerical distro or operating system I know of, Linux, BSD, Solaris or otherwise, currently offers an out-of-the-box module support for virtualization in their install initrd, for various licensing and technical reasons (the open-vm-tools effort would likely solve this in the near future). OHAI Trolls who can't read: YES YOU *CAN* PXEBOOT w/VMXNET (NOT VMXNET-ENHANCED), BUT YOUR INSTALL FAILS BECAUSE IT CAN'T FIND A "DRIVER" FOR 'VMXNET'.

I know of tru's efforts and others on this front and I really appreciate the knowledge they have brought to the table, but I feel that it's about time that some dedicated entity step in and 'solve' this problem (we've written kickstart+firstboot-type scripts that eliminate the 'work' of managing hundreds of VMs and their installation of tools, but through the whole process we have wished for Our Favorite Community Enterprise OS to support this stuff OOB so we didn't need to do the work. Lazy Sysadmins are lazy). One man cannot keep such a beast up to date; it needs to be a dedicated effort or project. The complexity of 'doing VMs right' is getting to the point where it requires a virtualization expert to be staffed (consider the ever-present Time Sync Issue, which requires customization on the Host configuration AND the Guest OS). This is unacceptable; we either need virtualization software that is a little more clever or we have to step in on the application layer and have the OS live closer to the host hardware. Also, there's a great deal of untapped potential in having at least a greater set of conservative message passing between the Guest OS and Host / VM Infrastructure, shared storage amongst different hosts over the attached SAN, etc etc... skies the limit.

This might be a direction that Big Mommy RedHat should/would eventually take. I'm not a big fan of CentOS mini-forks of kernels myself (e.g. centosplus, -vm, etc), since it kinda defeats the 'point' if you know what I mean.

/me crosses fingers that Sun + InnoTek makes a really kickass VirtualBox for the Enterprise that develops a much better solution, or better yet, a *truly* embedded virtualization solution (Hey, anyone from Sun Engineering on this mailing list? Imagine memory and storage pooling on the hardware layer over something like infiniband, guys).

Then again, cloud computing, future is bright, sunglasses + igloos, teotwawki, lol.

- allen tsang

Daniel de Kok wrote:

On Mon, May 5, 2008 at 5:13 AM, Allen Tsangatsang@advance.net wrote:

...

Speaking of which, I was talking to some friends about building a fully paravirtualized rhel/centos that works with xen, vmware, virtualbox, etc. Do you guys feel like that's a product you would consider using?

I suppose you mean "fully virtualized", since VMWare and VirtualBox do not paravirtualize? If so, True provides VMWare images of CentOS 4 and 5. They should be easy to modify for VirtualBox and qemu as well:

http://dev.centos.org/~tru/vmware/

Take care, Daniel _______________________________________________ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt

On Mon, May 5, 2008 at 11:06 AM, Allen Tsang atsang@advance.net wrote:

By Paravirtualization, I mean the installation of "tools" or "guest additions" type packages, which present virtual interfaces to the guest OS. So in VMware, a component of this would mean setting "ethernet0.virtualDev = vmxnet", and having the tools modules pre-installed. A fully virtualized OS for VMware would support all that crap.

Let's call these paravirtualized drivers ;), because the system itself is fully virtualized under VMWare and VirtualBox, that's why they can run unmodified operating systems.

I know of tru's efforts and others on this front and I really appreciate the knowledge they have brought to the table, but I feel that it's about time that some dedicated entity step in and 'solve' this problem

Can you concretely define the problem, and your proposed solution? I am not sure what you are aiming for, and what Tru's images are missing that you are looking for.

One man cannot keep such a beast up to date; it needs to be a dedicated effort or project.

I think Tru is doing a great job this far. The thing is that we could make a project out of any problem, but that in practice all work is done by one or a few dedicated people. Look at the i586 project: a lot of people say they need CentOS-5 for i586, some people volunteer, some people insist on creating a subproject for this, but in practice nothing happens until someone single-handedly gets the effort going.

To me, it seems best to send suggestions to Tru if something is missing. If you want to create something different, write a proposal and send it to this list (if it is related to virtualization, -devel otherwise).

Take care, Daniel

It might help to read what I actually wrote. You are making a lot of presumptions that are all not true and which you would know if you had really read what I wrote.

Speaking of which, I was talking to some friends about building a fully paravirtualized rhel/centos that works with xen, vmware, virtualbox, etc. Do you guys feel like that's a product you would consider using?

Would you please consider not hijacking threads with completely unrelated topics?

Kai

On Mon, May 5, 2008 at 7:23 PM, Allen Tsang atsang@advance.net wrote:

Don't let your hurt ego get in the way of the facts.

With the list moderator hat on:

Please quit the personal attacks. The subject of this list is virtualization, not psychology ;). Besides that, top-posting is not really appreciated, since it tends to mess with our brain-wiring.

Thanks, Daniel

Allen Tsang wrote:

Also just to review, clocksource=acpi_pm should be used in conjunction with the tools.synctime = "true" flag in your vmx file. The combination of the two settings prevents time from going into the future from too many ticks, and synctime corrects slow clocks, which leads to a much, much better clock sync.

That (clocksource=acpi_pm) is ONLY the best method if your clock is running SLOWER than normal .. and in fact, a "clocksource=pit" is probably the best solution for a clock that is running FASTER than normal. If the clock is running FASTER than normal, also getting the max frequency (host.cpukHz) set per these instructions is important:

http://blog.autoedification.com/2006/11/vmware-guest-clock-runs-fast.html

The /proc/ location in that article is not correct for centos4 or centos5, but you can probably get the correct info for frequency most of the time from /proc/cpuinfo, dmesg, or dmidecode.

We'll have to wait until someone figures out a clever way to tie VM clock ticks to a multiplexed physical clock source; until then, clocksync will always be a problem without a complete solution (read up on it). This is as close as it gets!

There is a potential fix in the Real Time Kernel () that Red Hat has released as part of their beta MRG release in that it might the support tickless option. I have not had time to look at this solution yet, but the kernel SRPM is here if someone wants to:

ftp://ftp.redhat.com/pub/redhat/linux/beta/MRG/RHEL-5/src/

<snip>

Thanks, Johnny Hughes

On Mon, May 5, 2008 at 10:39 AM, Allen Tsang atsang@advance.net wrote:

I've tried almost everything else under the sun with a number of test VMs on our ESX setup, and the best combination of low idle load and accurate timekeeping was achieved using that combination of settings. Yes, ntpd is off. Feel free to give it a shot; I don't take credit for it, I found notes about this on the CentOS bug tracker (http://bugs.centos.org/view.php?id=2189). Like other fine folks have mentioned on this list (much to their amusement likely, now), clocksource=pit is nice, but it doesn't work with divider=10 and hangs on boot.

As you can see in that bug tracker, we have spent a lot of time to come up with 100Hz kernels, kernel-vm. These kernels do not have problems with clocksource=pit. Until this bug is eliminated upstream, I think kernel-vm offered by CentOS may be the best solution (shameless advertisement). I just hope it won't take long for the upstream developers to find a fix for the problem associated with the divider= option.

Akemi / toracat