What helped a lot for me is to increase the read ahead drastically on
guests:
# Set read-ahead for optimal disk I/O
blockdev --setra 16384 /dev/vda
I optained this value by repeatedly timing copies and this was, for my
installation at least, optimal.
-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-
Eskimo North Linux Friendly Internet Access, Shell Accounts, and Hosting.
Knowledgeable human assistance, not telephone trees or script readers.
See our web site: http://www.eskimo.com/ (206) 812-0051 or (800) 246-6874.
On Thu, 20 Feb 2014, Zoltan Frombach wrote:
> Date: Thu, 20 Feb 2014 19:01:10 +0100
> From: Zoltan Frombach <zoltan at frombach.com>
> Reply-To: Discussion about the virtualization on CentOS
> <centos-virt at centos.org>
> To: Discussion about the virtualization on CentOS <centos-virt at centos.org>
> Subject: Re: [CentOS-virt] disk io in guests causes soft lockups in guests and
> host processes
>
> I experienced similar issues when disk images of virtual machines were
> stored in qcow/qcow2 files instead of logical volumes (LVM).
> Using LVM gives you way better I/O performace than using qcow files.
>
> Also very important: when you partition your disk drive(s) make sure
> that partitions are properly aligned to the physical allocation block
> size of the hard drive you use. Let's say your hard drive uses 4k
> sectors then every partition you create must start at a 4k boundary. If
> your partitions are mis-aligned then you'll get terrible disk I/O
> performace, just like the one you have described.
> For more info see, for example:
> http://www.thomas-krenn.com/en/wiki/Partition_Alignment
>
> You can also tweak Linux to get better KVM performance. For more info
> you can check out these documents:
> http://pic.dhe.ibm.com/infocenter/lnxinfo/v3r0m0/topic/liaat/liaatbestpractices_pdf.pdf
> http://www.novell.com/docrep/2013/05/kvm_virtualized_io_performance.pdf
>
> Zoltan
>
> On 2/20/2014 5:53 PM, Dennis Jacobfeuerborn wrote:
>> Hi,
>> I have a strange phenomenon that I cannot readily explain so I wonder if
>> anyone here can shed a light on this.
>>
>> The host system is a Dell r815 with 64 cores and 256G ram and has centos
>> 6 installed. The five guests are also running centos 6 and are running
>> as a hadoop cluster. The problem is that I see disk-io spikes in the
>> vm's which then cause soft lockups in the guest but I also see hanging
>> processes on the host as if the entire machine locks up for 30-60 seconds.
>>
>> Now I know that having all cluster members running on the same system
>> isn't efficient and that I cannot expect good performance but what I was
>> not expecting is that a guest make host processes hang.
>> Does anyone have an idea what the issue could be here or how I can find
>> out what cause for this behavior is?
>>
>> Regards,
>> Dennis
>> _______________________________________________
>> CentOS-virt mailing list
>> CentOS-virt at centos.org
>> http://lists.centos.org/mailman/listinfo/centos-virt
>>
>
> _______________________________________________
> CentOS-virt mailing list
> CentOS-virt at centos.org
> http://lists.centos.org/mailman/listinfo/centos-virt
>