<html><head></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; "><br><div><div>On Nov 21, 2013, at 10:48 AM, Digimer wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><div>What you do in the VMs does not impact the hosts, so I didn't speak to<br>that. Having the bridge, interfaces, switches and vnets at 9000 (for<br>example) doesn't immediately enable large frames in the virtual servers.<br>It simply means that all of the links between the VM and other devices<br>on the network are ready for JFs.<br><br>Imagine this;<br><br>{real switch}<br> |<br>{ethX + ethY}<br> |<br>{bondX}<br> |<br>{vbr0}<br> |<br>{vnetX}<br> |<br>{VM's eth0}<br><br>All of these devices need to have their MTU set to your desires value.<br>If any one of these is still 1500, then only standard frames will be<br>able to traverse them.<br><br>* real switch; Log into it and make sure jumbo frames are enabled<br><br>* ethX + ethY; If you are using bonding, be sure both/all slaved<br>interfaces are set to use a large frame.<br><br>* bondX; Again if you use a bond, make sure the bondX interface has a<br>large frame.<br><br>* vbr0; The bridge can not be set to a specific MTU size. It will use<br>the lowest MTU of the various devices / interfaces connected to it.<br><br>* vnetX; These are the "virtual network cables" that are used to "plug<br>in" a VM's interface to the bridge. This is not new by any means. In the<br>real world, network cables don't have setable MTUs of course. In the<br>virtual world though, they do. These interfaces are spontaneously<br>created and destroyed as VMs come and go. This is what the udev rule is<br>for because these "virtual network cables" don't have traditional<br>ifcfg-X files.<br><br>* VM's eth0; This is the (emulated) network card in your virtual server.<br>If you told the hypervisor to replicate an e1000 intel card or use the<br>virtio-net driver, you can set a large MTU. However, if you used<br>something like an emulated realtek card, those don't support jumbo<br>frames, so their emulated counterparts will not support large frames either.<br><br>hth<br><br>digimer<br><br>On 21/11/13 13:32, Nico Kadel-Garcia wrote:<br><blockquote type="cite">I was under the impression that the relevant MTU settings were on the<br></blockquote><blockquote type="cite">*node's* local ifcfg-eth* configurations. Did something change with<br></blockquote><blockquote type="cite">KVM internal networking in the last year?<br></blockquote><blockquote type="cite"><br></blockquote><blockquote type="cite">On Thu, Nov 21, 2013 at 1:03 PM, Digimer <<a href="mailto:lists@alteeve.ca">lists@alteeve.ca</a>> wrote:<br></blockquote><blockquote type="cite"><blockquote type="cite">The problem is that there are no ifcfg-vnetX config files. They are<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">dynamically created as VMs are created or migrated to a node. You could<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">manually (or via script) change the MTU, but that would mean that the<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">MTU on the bridge would drop momentarily when new VMs start. This could<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">break network traffic for any existing VMs (or real devices) using large<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">frames.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">I'm not a fan of udev either, but in this case, it is the best option.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">Of course, I am certainly open to hearing alternative methods if they exist.<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite">On 21/11/13 08:39, Nico Kadel-Garcia wrote:<br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Stay out of udev if you can. It's often overwritten by component<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">addition and manipulation MTU is parsed, and overridden, by options in<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">/etc/sysconfig/network-scripts/ifcfg-[device]. I find it much safer to<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">read and manage there, and if new devices are added or replaced, the<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">behavior is dominated by the "HWADDR" associated config files there,<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">no matter what "udev" thinks the device number or name should be..<br></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><snip><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Another update;<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> To make sure the VMs' vnetX devices are created with a larger MTU, you<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">*sill* need to update udev[1].<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> Append to /etc/udev/rules.d/70-persistent-net.rules;<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">====<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"># Make all VMs' vnetX devices come up with an MTU of 9000.<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">SUBSYSTEM=="net", ACTION=="add", KERNEL=="vnet*", ATTR{mtu}="9000"<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">====<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"> Assuming you find that you can use an MTU of '9000', of course. No<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">need to reboot or even restart networking. Just add that line and then<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">provision/boot your VMs. If the VMs are already running, you can adjust<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">the MTU of the existing 'vnetX' devices with:<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">ifconfig vnetX mtu 9000<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Cheers!<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">PS - Credit for the udev rule:<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><a href="http://linuxaleph.blogspot.ca/2013/01/how-to-network-jumbo-frames-to-kvm-guest.html">http://linuxaleph.blogspot.ca/2013/01/how-to-network-jumbo-frames-to-kvm-guest.html</a><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">--<br></blockquote></blockquote></blockquote></blockquote><blockquote type="cite"><blockquote type="cite"><blockquote type="cite"><blockquote type="cite">Digimer<br></blockquote></blockquote></blockquote></blockquote></div></blockquote><div><br></div></div>Hi,<div><br></div><div>I seem to lack a vnet to bridge device.</div><div><br></div><div>When I go to change my interface on the VM using the GUI, I do not see an option for "Host device vnet# (Bridge 'br6')</div><div><br></div><div>Instead I see "host device eth6 (Bridge 'br6') So before creating one via;</div><div><br></div><div>brctl addif...</div><div><br></div><div>Let me explain my config;</div><div><br></div><div>eth0 - standard MTU</div><div>eth1 - disabled</div><div>*eth6 - 10Gb at jumbo</div><div>* This card was added after KVM was setup and running.</div><div><br></div><div>My ifconfig output;</div><div><br></div><div><br></div><div><div>br0 Link encap:Ethernet HWaddr 00:25:90:63:9F:7A </div><div> inet addr:10.0.10.218 Bcast:10.0.255.255 Mask:255.255.0.0</div><div> inet6 addr: fe80::225:90ff:fe63:9f7a/64 Scope:Link</div><div> UP BROADCAST RUNNING MULTICAST <font class="Apple-style-span" color="#3930ff">MTU:1500</font> Metric:1</div><div> RX packets:754670 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:43162 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:0 </div><div> RX bytes:160904242 (153.4 MiB) TX bytes:51752758 (49.3 MiB)</div><div><br></div><div>br6 Link encap:Ethernet HWaddr 00:05:33:48:7B:29 </div><div> inet addr:10.0.10.220 Bcast:10.0.255.255 Mask:255.255.0.0</div><div> inet6 addr: fe80::205:33ff:fe48:7b29/64 Scope:Link</div><div> UP BROADCAST RUNNING MULTICAST <font class="Apple-style-span" color="#ff0000">MTU:9000</font> Metric:1</div><div> RX packets:4130 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:11150 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:0 </div><div> RX bytes:131498 (128.4 KiB) TX bytes:513156 (501.1 KiB)</div><div><br></div><div>eth0 Link encap:Ethernet HWaddr 00:25:90:63:9F:7A </div><div> inet6 addr: fe80::225:90ff:fe63:9f7a/64 Scope:Link</div><div> UP BROADCAST RUNNING MULTICAST <font class="Apple-style-span" color="#3b2eff">MTU:1500</font> Metric:1</div><div> RX packets:3379929 errors:18 dropped:0 overruns:0 frame:18</div><div> TX packets:3565007 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:1000 </div><div> RX bytes:840911383 (801.9 MiB) TX bytes:3519831013 (3.2 GiB)</div><div> Memory:fbbe0000-fbc00000 </div><div><br></div><div>eth6 Link encap:Ethernet HWaddr 00:05:33:48:7B:29 </div><div> UP BROADCAST RUNNING MULTICAST <font class="Apple-style-span" color="#ff0000">MTU:9000</font> Metric:1</div><div> RX packets:0 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:1000 </div><div> RX bytes:0 (0.0 b) TX bytes:0 (0.0 b)</div><div> Memory:fbd40000-fbd7ffff </div><div><br></div><div>lo Link encap:Local Loopback </div><div> inet addr:127.0.0.1 Mask:255.0.0.0</div><div> inet6 addr: ::1/128 Scope:Host</div><div> UP LOOPBACK RUNNING MTU:16436 Metric:1</div><div> RX packets:185130 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:185130 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:0 </div><div> RX bytes:138905226 (132.4 MiB) TX bytes:138905226 (132.4 MiB)</div><div><br></div><div>virbr0 Link encap:Ethernet HWaddr 52:54:00:CE:7A:65 </div><div> inet addr:192.168.122.1 Bcast:192.168.122.255 Mask:255.255.255.0</div><div> UP BROADCAST RUNNING MULTICAST <font class="Apple-style-span" color="#3b1bff">MTU:1500</font> Metric:1</div><div> RX packets:0 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:11139 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:0 </div><div> RX bytes:0 (0.0 b) TX bytes:512410 (500.4 KiB)</div><div><br></div><div>vnet0 Link encap:Ethernet HWaddr FE:30:48:7E:65:72 </div><div> inet6 addr: fe80::fc30:48ff:fe7e:6572/64 Scope:Link</div><div> UP BROADCAST RUNNING MULTICAST <font class="Apple-style-span" color="#ff0000"> MTU:9000</font> Metric:1</div><div> RX packets:1045 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:697730 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:500 </div><div> RX bytes:119723 (116.9 KiB) TX bytes:175334262 (167.2 MiB)</div><div><br></div><div>vnet1 Link encap:Ethernet HWaddr FE:16:36:0E:E7:F4 </div><div> inet6 addr: fe80::fc16:36ff:fe0e:e7f4/64 Scope:Link</div><div> UP BROADCAST RUNNING MULTICAST <font class="Apple-style-span" color="#ff0000">MTU:9000</font> Metric:1</div><div> RX packets:3494450 errors:0 dropped:0 overruns:0 frame:0</div><div> TX packets:3243369 errors:0 dropped:0 overruns:0 carrier:0</div><div> collisions:0 txqueuelen:500 </div><div> RX bytes:3466241191 (3.2 GiB) TX bytes:822212316 (784.1 MiB)</div></div><div><br></div><div>brctl show;</div><div><br></div><div><div>bridge name<span class="Apple-tab-span" style="white-space:pre"> </span>bridge id<span class="Apple-tab-span" style="white-space:pre"> </span>STP enabled<span class="Apple-tab-span" style="white-space:pre"> </span>interfaces</div><div>br0<span class="Apple-tab-span" style="white-space:pre"> </span>8000.002590639f7a<span class="Apple-tab-span" style="white-space:pre"> </span>no<span class="Apple-tab-span" style="white-space:pre"> </span>eth0</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>vnet0</div><div><span class="Apple-tab-span" style="white-space:pre"> </span>vnet1</div><div>br6<span class="Apple-tab-span" style="white-space:pre"> </span>8000.000533487b29<span class="Apple-tab-span" style="white-space:pre"> </span>no<span class="Apple-tab-span" style="white-space:pre"> </span>eth6</div><div>virbr0<span class="Apple-tab-span" style="white-space:pre"> </span>8000.525400ce7a65<span class="Apple-tab-span" style="white-space:pre"> </span>yes<span class="Apple-tab-span" style="white-space:pre"> </span>virbr0-nic</div></div><div><br></div><div><br></div><div>Should I have a virbr6?</div><div><br></div><div>I'm obviously pretty lost and must admin I sorta hate bridging in KVM.</div><div><br></div><div>- aurf</div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div></body></html>