[CentOS-virt] Xen vs. iSCSI
Luke S Crawford
lsc at prgmr.com
Tue Jun 16 19:59:54 UTC 2009
Bill McGonigle <bill at bfccomputing.com> writes:
> On 06/15/2009 11:33 PM, Luke S Crawford wrote:
> > xm sched-credit -d 0 60000
> Ah, ha! This appears to work. I didn't need to reserve a CPU for the
> dom0 (knock on wood). Much obliged, Luke.
> I'm academically curious, though - I seem to have created a CPU deadlock
> of some sort, yet in 'xm top' none of the CPU's were pegged. I've got
> no reason to not give dom0 utmost priority - that makes perfect sense to
> me - but I'm surprised the Xen scheduler would allow me to get into this
> situation by default.
My understanding of this is entirely janitor-level, but I believe what you
are seeing is that the dom0 has exhausted it's 'credits' and so if a
DomU wants the CPU the dom0 gets kicked off the cpu, waits a timeslice
(I think timeslices are on the order of tens of millaseconds... I've
read 60ms, which is quite a long time in terms of sending a packet to a
nearby storage box.) then gets back on the CPU.
This is why I'm always loath to give more than 1 or 2 vcpus to my DomUs,
and why I always reserve cpu0 for the dom0; that way, the domu
can pass a packet to the dom0 which can process it and send it out
without waiting. the domU and the Dom0 can run at the same time.
If you look (xm sched-credit -d 0) you will see that xen assigns
all DomUs a default priority of 256. It does not assign a higher priority
to the Dom0. I assume this is because the xen people very much have
an attitude of 'well, I wrote this nice hypervisor. You set it up.'
Which is fine with me, as while I can set it up, there's no way I could
have written the nice hypervisor. But yeah, I see no reason not to
default to giving the dom0 as much cpu as it wants; if the dom0 is
unhappy, everyone is unhappy.
It seems like the sort of thing RHEL could do. (well, that and
increasing the default dom0-min-mem to something that doesn't
crash the dom0.)
The 'xm sched-credit -d 0 60000' line is in the /etc/rc.local
of all the Xen hosts I administer. It helps a lot, even when you
use local disk.
I have seen this problem without using iscsi, when the DomUs are heavily
loaded. I get 'stutter' on the command line and dropped packets
on the interface counters. It's irritating, because without iscsi, the
problem is usually rare and difficult to reproduce.
More information about the CentOS-virt