Hi Daniel,
I also got a full time work so I know how it is, on what URL can I found the bug tracker?
I think there are 2 bugs, one being the CPU lockup and the 2nd where Xen hangs under high load, this is what I saw when it hung last time:
printk: 220 messages suppressed. httpd invoked oom-killer: gfp_mask=0x201d2, order=0, oomkilladj=0
Call Trace: [<ffffffff802aee7d>] out_of_memory+0x4e/0x1d3 [<ffffffff8020efe8>] __alloc_pages+0x229/0x2b2 [<ffffffff8021298b>] __do_page_cache_readahead+0xd0/0x21c [<ffffffff802284d8>] sync_page+0x0/0x42 [<ffffffff88082c61>] :dm_mod:dm_any_congested+0x38/0x3f [<ffffffff80213240>] filemap_nopage+0x148/0x322 [<ffffffff80208b94>] __handle_mm_fault+0x3da/0xf46 [<ffffffff802606e9>] _spin_lock_irqsave+0x9/0x14 [<ffffffff80262fc8>] do_page_fault+0xe48/0x11dc [<ffffffff8022c50c>] mntput_no_expire+0x19/0x89 [<ffffffff80245fbe>] sys_chdir+0x55/0x62 [<ffffffff8025cb6f>] error_exit+0x0/0x6e
DMA per-cpu: cpu 0 hot: high 186, batch 31 used:170 cpu 0 cold: high 62, batch 15 used:55 cpu 1 hot: high 186, batch 31 used:28 cpu 1 cold: high 62, batch 15 used:37 cpu 2 hot: high 186, batch 31 used:24 cpu 2 cold: high 62, batch 15 used:47 cpu 3 hot: high 186, batch 31 used:14 cpu 3 cold: high 62, batch 15 used:52
Call Trace: [<ffffffff802aee7d>] out_of_memory+0x4e/0x1d3 [<ffffffff8020efe8>] __alloc_pages+0x229/0x2b2 [<ffffffff8021298b>] __do_page_cache_readahead+0xd0/0x21c [<ffffffff8025f528>] __wait_on_bit_lock+0x5b/0x66 [<ffffffff88082c61>] :dm_mod:dm_any_congested+0x38/0x3f [<ffffffff80213240>] filemap_nopage+0x148/0x322 [<ffffffff80208b94>] __handle_mm_fault+0x3da/0xf46 [<ffffffff802606e9>] _spin_lock_irqsave+0x9/0x14 [<ffffffff80262fc8>] do_page_fault+0xe48/0x11dc [<ffffffff80233a31>] do_setitimer+0x45f/0x4c7 [<ffffffff80245fbe>] sys_chdir+0x55/0x62 [<ffffffff8025cb6f>] error_exit+0x0/0x6e
DMA per-cpu: cpu 0 hot: high 186, batch 31 used:170 cpu 0 cold: high 62, batch 15 used:55 cpu 1 hot: high 186, batch 31 used:28 cpu 1 cold: high 62, batch 15 used:37 cpu 2 hot: high 186, batch 31 used:24 cpu 2 cold: high 62, batch 15 used:47 cpu 3 hot: high 186, batch 31 used:14 cpu 3 cold: high 62, batch 15 used:52 cpu 4 hot: high 186, batch 31 used:25 cpu 4 cold: high 62, batch 15 used:13 cpu 5 hot: high 186, batch 31 used:21 cpu 5 cold: high 62, batch 15 used:54 cpu 6 hot: high 186, batch 31 used:17 cpu 6 cold: high 62, batch 15 used:45 cpu 7 hot: high 186, batch 31 used:17 cpu 7 cold: high 62, batch 15 used:45 DMA32 per-cpu: empty Normal per-cpu: empty HighMem per-cpu: empty Free pages: 6020kB (0kB HighMem) Active:300020 inactive:204495 dirty:0 writeback:0 unstable:0 free:1505 slab:7395 mapped:2 pagetables:24274 DMA free:6020kB min:6020kB low:7524kB high:9028kB active:1200080kB inactive:817980kB present:2265088kB pages_scanned:19172661 all_unreclaimable? yes lowmem_reserve[]: 0 0 0 0 DMA32 free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 Normal free:0kB min:0kB low:0kB high:0kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 HighMem free:0kB min:128kB low:128kB high:128kB active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 0 0 DMA: 65*4kB 0*8kB 0*16kB 8*32kB 2*64kB 0*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 1*4096kB = 6020kB DMA32: empty Normal: empty HighMem: empty Swap cache: add 217269, delete 217269, find 2978/10987, race 0+1 Free swap = 0kB Total swap = 557048kB
All swap is used so no memory free, this would make a non virtual box very slow (almost dead), but not crash like this so could it be something wrong moving swap pages back and forth that causes this problem..?
Regards, Nicolas Sahlqvist CouchSurfing.com
On 10/9/07, Daniel de Kok danieldk@pobox.com wrote:
On Tue, 2007-10-09 at 10:47 +0200, Nicolas Sahlqvist wrote:
So you are saying that we should run more zones with less CPU's to avoid the problem? Well, the budget for RAM would not go too well with that so is there any work done on a fix that you are aware of?
Well, I was rather wondering if it is the same bug. Even if so, it should really be filed in our and the upstream bug trackers. I should have done that myself, but have had little time to do so.
-- Daniel
CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt