Bill McGonigle <bill at bfccomputing.com> writes: > On 06/15/2009 11:33 PM, Luke S Crawford wrote: > > xm sched-credit -d 0 60000 > > Ah, ha! This appears to work. I didn't need to reserve a CPU for the > dom0 (knock on wood). Much obliged, Luke. > > I'm academically curious, though - I seem to have created a CPU deadlock > of some sort, yet in 'xm top' none of the CPU's were pegged. I've got > no reason to not give dom0 utmost priority - that makes perfect sense to > me - but I'm surprised the Xen scheduler would allow me to get into this > situation by default. My understanding of this is entirely janitor-level, but I believe what you are seeing is that the dom0 has exhausted it's 'credits' and so if a DomU wants the CPU the dom0 gets kicked off the cpu, waits a timeslice (I think timeslices are on the order of tens of millaseconds... I've read 60ms, which is quite a long time in terms of sending a packet to a nearby storage box.) then gets back on the CPU. This is why I'm always loath to give more than 1 or 2 vcpus to my DomUs, and why I always reserve cpu0 for the dom0; that way, the domu can pass a packet to the dom0 which can process it and send it out without waiting. the domU and the Dom0 can run at the same time. If you look (xm sched-credit -d 0) you will see that xen assigns all DomUs a default priority of 256. It does not assign a higher priority to the Dom0. I assume this is because the xen people very much have an attitude of 'well, I wrote this nice hypervisor. You set it up.' Which is fine with me, as while I can set it up, there's no way I could have written the nice hypervisor. But yeah, I see no reason not to default to giving the dom0 as much cpu as it wants; if the dom0 is unhappy, everyone is unhappy. It seems like the sort of thing RHEL could do. (well, that and increasing the default dom0-min-mem to something that doesn't crash the dom0.) The 'xm sched-credit -d 0 60000' line is in the /etc/rc.local of all the Xen hosts I administer. It helps a lot, even when you use local disk. I have seen this problem without using iscsi, when the DomUs are heavily loaded. I get 'stutter' on the command line and dropped packets on the interface counters. It's irritating, because without iscsi, the problem is usually rare and difficult to reproduce.