[CentOS-virt] Libvirt NAT-based network 2x-4x faster than Libvirt routed network?

Tue May 25 13:34:17 UTC 2010
Kenni Lund <kenni at kelu.dk>

Hi

Is it expected behaviour that a libvirt NAT-based network is faster
than a libvirt routed network?

I would guess that the routed network would be the fastest one in all
cases, but I'm seeing the opposite in my setup. On a 100mbit internet
connection, the routed network tops at 4mbyte/sec, while the nat-based
network goes all the way to 10mbyte/sec. I've spend the last 6 hours
on trying to figure out the cause of this, changing settings,
reinstalling the system, but the problem persists and I can't find the
cause of it.

It is a clean and fairly simple setup:
- Clean installation of CentOS 5.5
- Installation of kvm+libvirt+virt-manager+xorg
- iptables service disabled
- 1 public IP for the host (xxx.yyy.zzz.215) on a /32 subnet with
xxx.yyy.zzz.193 as gateway
- 3 public IP for the routed network (xxx.yyy.zzz.251-253) on a /26
subnet with the same gateway as the host
- 1 virtual machine with either the NAT or the routed libvirt network assigned

I have no networking connection errors or similar, it works perfectly
with both the NAT and the routed network, but the routed network is
just 2-4 times slower than the NAT one.

Do you have any suggestions of what the problem might be? When I look
at the iptables rules generated by libvirt, and the routing table,
everything looks fine AFAICT. Traceroute from guest doesn't reveal
anything either. Ping times from the two networks are similar, so the
main issue seems to be throughput.

Any help or pointers to what I should look at, is highly appreciated... :)

I've copy pasted some of my relevant system configuration into pastebin:
http://pastebin.com/jtTrHLqA

Thank you,
Kenni