[CentOS-virt] disk io in guests causes soft lockups in guests and host processes

Thu Feb 20 16:53:47 UTC 2014
Dennis Jacobfeuerborn <dennisml at conversis.de>

I have a strange phenomenon that I cannot readily explain so I wonder if 
anyone here can shed a light on this.

The host system is a Dell r815 with 64 cores and 256G ram and has centos 
6 installed. The five guests are also running centos 6 and are running 
as a hadoop cluster. The problem is that I see disk-io spikes in the 
vm's which then cause soft lockups in the guest but I also see hanging 
processes on the host as if the entire machine locks up for 30-60 seconds.

Now I know that having all cluster members running on the same system 
isn't efficient and that I cannot expect good performance but what I was 
not expecting is that a guest make host processes hang.
Does anyone have an idea what the issue could be here or how I can find 
out what cause for this behavior is?