I have a CentOS 5.1 dom0 running on a machine with two cores and 4gb of memory. It runs three 4.6 domUs and one 5.1 domU. Just out of the blue the other night all the 4.6 domUs crashed, but the dom0 stayed up. I ran xm console (domU-name) and got the Oops information below. I didn't get anything from the 5.1 domU. I think it had been rebooted a few days before and hadn't come up because of a initrd issue.

First 4.6 domU:

kernel BUG at arch/i386/mm/pgtable-xen.c:306! invalid operand: 0000 [#1] SMP Modules linked in: md5 ipv6 sunrpc dm_mirror dm_mod xennet ext3 jbd xenblk CPU: 0 EIP: 0061:[<c011163a>] Not tainted VLI EFLAGS: 00010282 (2.6.9-55.0.12.ELxenU) EIP is at pgd_ctor+0x1d/0x26 eax: fffffff4 ebx: 00000000 ecx: f5392000 edx: 00000000 esi: c19fdd80 edi: e8f6a700 ebp: 00000001 esp: e0e45de4 ds: 007b es: 007b ss: 0068 Process crond (pid: 32004, threadinfo=e0e45000 task=eb4443f0) Stack: c0141bbd cc57b000 c19fdd80 00000001 cc57b000 e8f6a700 c19fdd80 c19fde40 c0141d3f c19fdd80 e8f6a700 00000001 c19fdd80 e8f6a700 cc57b000 00000010 00000001 000000d0 c1a1b080 0000000c c19fde08 c19fdd80 c0141f2e c19fdd80 Call Trace: [<c0141bbd>] cache_init_objs+0x35/0x56 [<c0141d3f>] cache_grow+0xfb/0x187 [<c0141f2e>] cache_alloc_refill+0x163/0x19c [<c0142149>] kmem_cache_alloc+0x67/0x97 [<c0111671>] pgd_alloc+0x17/0x336 [<c01199d4>] mm_init+0xd7/0x116 [<c01199e4>] mm_init+0xe7/0x116 [<c0119a3d>] mm_alloc+0x2a/0x31 [<c0163097>] do_execve+0x82/0x210 [<c0105d79>] sys_execve+0x2c/0x8e [<c010737f>] syscall_call+0x7/0xb Code: 74 02 66 a5 a8 01 74 01 a4 5e 5b 5e 5f c3 80 3d 04 07 2f c0 00 75 1c 6a 20 6a 00 ff 74 24 0c e8 ce 37 00 00 83 c4 0c 85 c0 74 08 <0f> 0b 32 01 ed 33 27 c0 c3 80 3d 04 07 2f c0 00 75 0d c7 44 24 <0>Fatal exception: panic in 5 seconds Kernel panic - not syncing: Fatal exception

Second 4.6 domU:

kernel BUG at arch/i386/mm/pgtable-xen.c:306! invalid operand: 0000 [#1] SMP Modules linked in: nfsd exportfs lockd nfs_acl md5 ipv6 sunrpc dm_mirror dm_mod xennet ext3 jbd xenblk CPU: 0 EIP: 0061:[<c011163a>] Not tainted VLI EFLAGS: 00010282 (2.6.9-55.0.12.ELxenU) EIP is at pgd_ctor+0x1d/0x26 eax: fffffff4 ebx: 00000000 ecx: f5392000 edx: 00000000 esi: c19fdd80 edi: e69f4c40 ebp: 00000001 esp: e7282d6c ds: 007b es: 007b ss: 0068 Process nrpe (pid: 1936, threadinfo=e7282000 task=e72cf770) Stack: c0141bbd c7ebb000 c19fdd80 00000001 c7ebb000 e69f4c40 c19fdd80 c19fde40 c0141d3f c19fdd80 e69f4c40 00000001 c19fdd80 e69f4c40 c7ebb000 00000010 00000001 000000d0 c1a1b080 0000000c c19fde08 c19fdd80 c0141f2e c19fdd80 Call Trace: [<c0141bbd>] cache_init_objs+0x35/0x56 [<c0141d3f>] cache_grow+0xfb/0x187 [<c0141f2e>] cache_alloc_refill+0x163/0x19c [<c0142149>] kmem_cache_alloc+0x67/0x97 [<c0111671>] pgd_alloc+0x17/0x336 [<c01199d4>] mm_init+0xd7/0x116 [<c01199e4>] mm_init+0xe7/0x116 [<c0119c8a>] copy_mm+0xbb/0x396 [<c0141f1f>] cache_alloc_refill+0x154/0x19c [<c011aa5a>] copy_process+0x6b5/0xb0b [<c011af9d>] do_fork+0x8a/0x16b [<c020f4c4>] sys_socketcall+0x113/0x202 [<c0105d2c>] sys_clone+0x24/0x28 [<c010737f>] syscall_call+0x7/0xb Code: 74 02 66 a5 a8 01 74 01 a4 5e 5b 5e 5f c3 80 3d 04 07 2f c0 00 75 1c 6a 20 6a 00 ff 74 24 0c e8 ce 37 00 00 83 c4 0c 85 c0 74 08 <0f> 0b 32 01 ed 33 27 c0 c3 80 3d 04 07 2f c0 00 75 0d c7 44 24 <0>Fatal exception: panic in 5 seconds Kernel panic - not syncing: Fatal exception

Third 4.6 domU: kernel BUG at arch/i386/mm/pgtable-xen.c:306! invalid operand: 0000 [#1] SMP Modules linked in: md5 ipv6 sunrpc dm_mirror dm_mod xennet ext3 jbd xenblk CPU: 0 EIP: 0061:[<c011163a>] Not tainted VLI EFLAGS: 00010282 (2.6.9-55.0.12.ELxenU) EIP is at pgd_ctor+0x1d/0x26 eax: fffffff4 ebx: 00000000 ecx: f5392000 edx: 00000000 esi: c15fdd80 edi: d3f312e0 ebp: 00000001 esp: ca9c4de4 ds: 007b es: 007b ss: 0068 Process qmail-rspawn (pid: 23442, threadinfo=ca9c4000 task=da68c2b0) Stack: c0141bbd ced8c000 c15fdd80 00000001 ced8c000 d3f312e0 c15fdd80 c15fde40 c0141d3f c15fdd80 d3f312e0 00000001 c15fdd80 d3f312e0 ced8c000 00000010 00000001 000000d0 c1613080 0000000c c15fde08 c15fdd80 c0141f2e c15fdd80 Call Trace: [<c0141bbd>] cache_init_objs+0x35/0x56 [<c0141d3f>] cache_grow+0xfb/0x187 [<c0141f2e>] cache_alloc_refill+0x163/0x19c [<c0142149>] kmem_cache_alloc+0x67/0x97 [<c0111671>] pgd_alloc+0x17/0x336 [<c01199d4>] mm_init+0xd7/0x116 [<c01199e4>] mm_init+0xe7/0x116 [<c0119a3d>] mm_alloc+0x2a/0x31 [<c0163097>] do_execve+0x82/0x210 [<c0105d79>] sys_execve+0x2c/0x8e [<c010737f>] syscall_call+0x7/0xb Code: 74 02 66 a5 a8 01 74 01 a4 5e 5b 5e 5f c3 80 3d 04 07 2f c0 00 75 1c 6a 20 6a 00 ff 74 24 0c e8 ce 37 00 00 83 c4 0c 85 c0 74 08 <0f> 0b 32 01 ed 33 27 c0 c3 80 3d 04 07 2f c0 00 75 0d c7 44 24 <0>Fatal exception: panic in 5 seconds Kernel panic - not syncing: Fatal exception