[CentOS-virt] proper bridging technoque

Thu Nov 21 22:15:13 UTC 2013
aurfalien <aurfalien at gmail.com>

On Nov 21, 2013, at 10:48 AM, Digimer wrote:

> What you do in the VMs does not impact the hosts, so I didn't speak to
> that. Having the bridge, interfaces, switches and vnets at 9000 (for
> example) doesn't immediately enable large frames in the virtual servers.
> It simply means that all of the links between the VM and other devices
> on the network are ready for JFs.
> 
> Imagine this;
> 
> {real switch}
> |
> {ethX + ethY}
> |
> {bondX}
> |
> {vbr0}
> |
> {vnetX}
> |
> {VM's eth0}
> 
> All of these devices need to have their MTU set to your desires value.
> If any one of these is still 1500, then only standard frames will be
> able to traverse them.
> 
> * real switch; Log into it and make sure jumbo frames are enabled
> 
> * ethX + ethY; If you are using bonding, be sure both/all slaved
> interfaces are set to use a large frame.
> 
> * bondX; Again if you use a bond, make sure the bondX interface has a
> large frame.
> 
> * vbr0; The bridge can not be set to a specific MTU size. It will use
> the lowest MTU of the various devices / interfaces connected to it.
> 
> * vnetX; These are the "virtual network cables" that are used to "plug
> in" a VM's interface to the bridge. This is not new by any means. In the
> real world, network cables don't have setable MTUs of course. In the
> virtual world though, they do. These interfaces are spontaneously
> created and destroyed as VMs come and go. This is what the udev rule is
> for because these "virtual network cables" don't have traditional
> ifcfg-X files.
> 
> * VM's eth0; This is the (emulated) network card in your virtual server.
> If you told the hypervisor to replicate an e1000 intel card or use the
> virtio-net driver, you can set a large MTU. However, if you used
> something like an emulated realtek card, those don't support jumbo
> frames, so their emulated counterparts will not support large frames either.
> 
> hth
> 
> digimer
> 
> On 21/11/13 13:32, Nico Kadel-Garcia wrote:
>> I was under the impression that the relevant MTU settings were on the
>> *node's* local ifcfg-eth* configurations. Did something change with
>> KVM internal networking in the last year?
>> 
>> On Thu, Nov 21, 2013 at 1:03 PM, Digimer <lists at alteeve.ca> wrote:
>>> The problem is that there are no ifcfg-vnetX config files. They are
>>> dynamically created as VMs are created or migrated to a node. You could
>>> manually (or via script) change the MTU, but that would mean that the
>>> MTU on the bridge would drop momentarily when new VMs start. This could
>>> break network traffic for any existing VMs (or real devices) using large
>>> frames.
>>> 
>>> I'm not a fan of udev either, but in this case, it is the best option.
>>> Of course, I am certainly open to hearing alternative methods if they exist.
>>> 
>>> On 21/11/13 08:39, Nico Kadel-Garcia wrote:
>>>> Stay out of udev if you can. It's often overwritten by component
>>>> addition and manipulation MTU is parsed, and overridden, by options in
>>>> /etc/sysconfig/network-scripts/ifcfg-[device]. I find it much safer to
>>>> read and manage there, and if new devices are added or replaced, the
>>>> behavior is dominated by the "HWADDR" associated config files there,
>>>> no matter what "udev" thinks the device number or name should be..
>>> 
>>> <snip>
>>> 
>>>>> 
>>>>> Another update;
>>>>> 
>>>>>  To make sure the VMs' vnetX devices are created with a larger MTU, you
>>>>> *sill* need to update udev[1].
>>>>> 
>>>>>  Append to /etc/udev/rules.d/70-persistent-net.rules;
>>>>> 
>>>>> ====
>>>>> # Make all VMs' vnetX devices come up with an MTU of 9000.
>>>>> SUBSYSTEM=="net", ACTION=="add", KERNEL=="vnet*", ATTR{mtu}="9000"
>>>>> ====
>>>>> 
>>>>>  Assuming you find that you can use an MTU of '9000', of course. No
>>>>> need to reboot or even restart networking. Just add that line and then
>>>>> provision/boot your VMs. If the VMs are already running, you can adjust
>>>>> the MTU of the existing 'vnetX' devices with:
>>>>> 
>>>>> ifconfig vnetX mtu 9000
>>>>> 
>>>>> Cheers!
>>>>> 
>>>>> PS - Credit for the udev rule:
>>>>> 
>>>>> http://linuxaleph.blogspot.ca/2013/01/how-to-network-jumbo-frames-to-kvm-guest.html
>>>>> 
>>>>> --
>>>>> Digimer

Hi,

I seem to lack a vnet to bridge device.

When I go to change my interface on the VM using the GUI, I do not see an option for "Host device vnet# (Bridge 'br6')

Instead I see "host device eth6 (Bridge 'br6')  So before creating one via;

brctl addif...

Let me explain my config;

eth0 - standard MTU
eth1 - disabled
*eth6 - 10Gb at jumbo
* This card was added after KVM was setup and running.

My ifconfig output;


br0       Link encap:Ethernet  HWaddr 00:25:90:63:9F:7A  
          inet addr:10.0.10.218  Bcast:10.0.255.255  Mask:255.255.0.0
          inet6 addr: fe80::225:90ff:fe63:9f7a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:754670 errors:0 dropped:0 overruns:0 frame:0
          TX packets:43162 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:160904242 (153.4 MiB)  TX bytes:51752758 (49.3 MiB)

br6       Link encap:Ethernet  HWaddr 00:05:33:48:7B:29  
          inet addr:10.0.10.220  Bcast:10.0.255.255  Mask:255.255.0.0
          inet6 addr: fe80::205:33ff:fe48:7b29/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:4130 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11150 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:131498 (128.4 KiB)  TX bytes:513156 (501.1 KiB)

eth0      Link encap:Ethernet  HWaddr 00:25:90:63:9F:7A  
          inet6 addr: fe80::225:90ff:fe63:9f7a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3379929 errors:18 dropped:0 overruns:0 frame:18
          TX packets:3565007 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:840911383 (801.9 MiB)  TX bytes:3519831013 (3.2 GiB)
          Memory:fbbe0000-fbc00000 

eth6      Link encap:Ethernet  HWaddr 00:05:33:48:7B:29  
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
          Memory:fbd40000-fbd7ffff 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:185130 errors:0 dropped:0 overruns:0 frame:0
          TX packets:185130 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:138905226 (132.4 MiB)  TX bytes:138905226 (132.4 MiB)

virbr0    Link encap:Ethernet  HWaddr 52:54:00:CE:7A:65  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11139 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:512410 (500.4 KiB)

vnet0     Link encap:Ethernet  HWaddr FE:30:48:7E:65:72  
          inet6 addr: fe80::fc30:48ff:fe7e:6572/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:1045 errors:0 dropped:0 overruns:0 frame:0
          TX packets:697730 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:119723 (116.9 KiB)  TX bytes:175334262 (167.2 MiB)

vnet1     Link encap:Ethernet  HWaddr FE:16:36:0E:E7:F4  
          inet6 addr: fe80::fc16:36ff:fe0e:e7f4/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:9000  Metric:1
          RX packets:3494450 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3243369 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:500 
          RX bytes:3466241191 (3.2 GiB)  TX bytes:822212316 (784.1 MiB)

brctl show;

bridge name	bridge id		STP enabled	interfaces
br0		8000.002590639f7a	no		eth0
							vnet0
							vnet1
br6		8000.000533487b29	no		eth6
virbr0		8000.525400ce7a65	yes		virbr0-nic


Should I have a virbr6?

I'm obviously pretty lost and must admin I sorta hate bridging in KVM.

- aurf







-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/centos-virt/attachments/20131121/e6140460/attachment-0004.html>