[CentOS-virt] CentOS 6 kvm disk write performance

[Julian price]

I have 2 similar servers. Since upgrading one from CentOS 5.5 to 6, disk 
write performance in kvm guest VMs is much worse.

There are many, many posts about optimising kvm, many mentioning disk 
performance in CentOS 5 vs 6.  I've tried various changes to speed up 
write performance, but northing's made a significant difference so far:

- Install virtio disk drivers in guest
- update the host software
- Update RAID firmware to latest version
- Switch the host disk scheduler to deadline
- Increase host RAM from 8GB to 24GB
- Increase guest RAM from 2GB to 4GB
- Try different kvm cache options
- Switch host from ext4 back to ext3
- Set noatime on the virtual disk image file
Note: There is no encryption or on-access virus scanner on any host or 
guest.

Below are some the block write figures in MB/s from bonnie++ with 
various configurations:

First, figures for the hosts show that the CentOS 6 server is faster:

54    CentOS 5 Host
50    CentOS 5 Host
69    CentOS 6 host
70    CentOS 6 host

Figures for a CentOS 6 guest running on the CentOS 5 host show that the 
performance hit is less than 50%:

30    CentOS 6 guest on CentOS 5 host with no optimisations
27    CentOS 6 guest on CentOS 5 host with no optimisations
32    CentOS 6 guest on CentOS 5 host with no optimisations

Here are the figures a CentOS 6 guest running on the CentOS 6 host with 
various optimisations.  Even with these optimisations, performance 
doesn't come close to the un-optimised guest running on the CentoOS 5 host:

 5   No optimisations (i.e. same configuration as on CentOS 5)
 4   deadline scheduler
 5   deadline scheduler
15   noatime,nodiratime
14   noatime,nodiratime
15   noatime
15   noatime + deadline scheduler
13   virtio
13   virtio
10   virtio + noatime
 9   virtio + noatime

The CentOS 6 server has a better RAID card, different disks and more 
RAM, which might account for the better CentOS 6 host performance.  But 
why might the guest write performance be so much worse?

Is this a known problem?  If so, what's the cause?    If not, is there a 
way to locate the problem rather than using trial and error?

Thanks,
Julian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.centos.org/pipermail/centos-virt/attachments/20120810/d7b4b057/attachment.html