I'm pretty consistently getting crashed KVM VMs under network I/O load when using the e1000 and virtio NIC models, with the follow message in dmesg:
qemu-kvm[8785]: segfault at d0 ip 00000031e3200a08 sp 0000000047784080 error 4 in libnvidia-tls.so.285.05.23[31e3200000+3000]
qemu-kvm[8791]: segfault at d0 ip 00000031e3200a08 sp 0000000049f89080 error 4 in libnvidia-tls.so.285.05.23[31e3200000+3000]
qemu-kvm[8792]: segfault at d0 ip 00000031e3200a08 sp 000000004a78a080 error 4 in libnvidia-tls.so.285.05.23[31e3200000+3000]
[<ffffffff8113b41b>] ? newseg+0xf0/0x213
[<ffffffff8113b32b>] ? newseg+0x0/0x213
These VMs are not performing any tasks with their display that I am aware of, and are not assigned GPUs. I'm aware that this isn't the latest nvidia driver, but I guess I don't understand why it's causing the problem in the first place.
The same thing also happens when using the virtio block driver under heavy load. Unfortunately this is forcing me to fall back to the default NIC drivers, which are very slow (~120Mbit/s VM<->VM vs ~460Mbit/s with virtio).
We're using vanilla kernel 2.6.32, with the following KVM-related packages:
kmod-kvm-83-239.el5.centos
kvm-qemu-img-83-239.el5.centos
kvm-83-239.el5.centos
libvirt-0.8.2-22.el5
Here is an example config (you can see where I commented out the model lines, because the VMs won't stay up):
<domain type='kvm'>
<name>myvm</name>
<memory>8388608</memory>
<currentMemory>8388608</currentMemory>
<vcpu cpuset="0-7">8</vcpu>
<os>
<type arch='x86_64' machine='rhel5.4.0'>hvm</type>
<boot dev='hd'/>
</os>
<features>
<acpi/>
<apic/>
<pae/>
</features>
<clock offset='utc'/>
<on_poweroff>destroy</on_poweroff>
<on_reboot>restart</on_reboot>
<on_crash>restart</on_crash>
<devices>
<emulator>/usr/libexec/qemu-kvm</emulator>
<disk type='file' device='disk'>
<driver name='qemu' cache='none' type='raw'/>
<source file='/vm/path/disk.raw'/>
<!-- <target dev='vda' bus='virtio'/> -->
<target dev='vda'/>
</disk>
<interface type='bridge'>
<mac address='mac1'/>
<source bridge='br0'/>
<!-- <model type='e1000'/> -->
</interface>
<interface type='bridge'>
<mac address='mac2'/>
<source bridge='br9'/>
<!-- <model type='e1000'/> -->
</interface>
<serial type='pty'>
<source path='/dev/pts/2'/>
<target port='0'/>
</serial>
<console type='pty' tty='/dev/pts/2'>
<source path='/dev/pts/2'/>
<target port='0'/>
</console>
<!-- <graphics type='vnc' autoport='yes'/> -->
</devices>
</domain>
Any help would be appreciated!
-Ben