Hello,
Presently I am running CentOS release 5.5 (Final). I am looking to setup bridging as I would like to setup some KVM virtual hosts on my system as a test lab. I am following the the instruction at this site
but I cannot figure out where I am going wrong and would be thankful if someone could point me in the right direction.
Here is what I have done:
Check bridge information with the following:
~ $ modprobe -v bridge
No issues or errors
~ $ cat /proc/modules | grep bridge bidge 91889 0 - Live 0xffffffff89247000
Check to ensure forwarding is turned on:
~ $ cat /proc/sys/net/ipv4/ip_forward 1
Checked that my interface are up and running (Was sure of this but did the check anyway):
~ $ ifconfig eth0 Link encap:Ethernet HWaddr 48:5B:39:2A:07:D5 inet addr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::4a5b:39ff:fe2a:7d5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1059 errors:0 dropped:0 overruns:0 frame:0 TX packets:1080 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:454226 (443.5 KiB) TX bytes:120584 (117.7 KiB) Interrupt:90 Base address:0x8400
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:92 errors:0 dropped:0 overruns:0 frame:0 TX packets:92 errors:0 dropped:0 overruns:0 carrier:0[Thu Nov 25
So now I begin to create the bridge form CLI as I want to make sure everything works before committing it to the config:
brctl addbr br0 ifconfig eth0 down ifconfig br0 192.168.1.100 up ifconfig eth0 0.0.0.0 up route add default gw 192.168.1.254
I check my interfaces and routing:
~ $ ifconfig br0 Link encap:Ethernet HWaddr 00:00:00:00:00:00 inet addr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:5 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:398 (398.0 b)
eth0 Link encap:Ethernet HWaddr 48:5B:39:2A:07:D5 inet6 addr: fe80::4a5b:39ff:fe2a:7d5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:64662 errors:0 dropped:0 overruns:0 frame:0 TX packets:63301 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:17699194 (16.8 MiB) TX bytes:7958063 (7.5 MiB) Interrupt:90 Base address:0x8400
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:211 errors:0 dropped:0 overruns:0 frame:0 TX packets:211 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:17346 (16.9 KiB) TX bytes:17346 (16.9 KiB)
~ $ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use
Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0 0.0.0.0 192.168.1.254 0.0.0.0 UG 0 0 0 br0
Time to test if ping works:
~ $ ping -c3 192.168.1.254 PING 192.168.1.254 (192.168.1.254) 56(84) bytes of data. ping: sendmsg: Operation not permitted ping: sendmsg: Operation not permitted ping: sendmsg: Operation not permitted
--- 192.168.1.254 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2000ms
I know the firewall is causing this issue so I stop the firewall:
~ $ service iptables stop Flushing firewall rules: [ OK ] Setting chains to policy ACCEPT: nat filter [ OK ] Unloading iptables modules: [ OK ]
Time to try ping again:
~ $ ping -c3 192.168.1.254 PING 192.168.1.254 (192.168.1.254) 56(84) bytes of data. From 192.168.1.100 icmp_seq=1 Destination Host Unreachable From 192.168.1.100 icmp_seq=2 Destination Host Unreachable From 192.168.1.100 icmp_seq=3 Destination Host Unreachable
--- 192.168.1.254 ping statistics --- 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 1999ms
To back out all I needed to do was:
ifconfig br0 down brctl delbr br0 service iptables start service netowrk restart
Everything is back to normal. I cannot figure out what am I missing here? Interfaces and routing look to be setup correctly. Is there something else I need to be looking at?
On Fri, Nov 26, 2010 at 2:15 PM, Robert Spangler mlists@zoominternet.net wrote:
Hello,
Presently I am running CentOS release 5.5 (Final). I am looking to setup bridging as I would like to setup some KVM virtual hosts on my system as a
Time to test if ping works:
~ $ ping -c3 192.168.1.254 PING 192.168.1.254 (192.168.1.254) 56(84) bytes of data. ping: sendmsg: Operation not permitted ping: sendmsg: Operation not permitted ping: sendmsg: Operation not permitted
Did you remember to brctl addif the regular interfaces?
On Friday 26 November 2010 12:22, Eduardo Grosclaude wrote:
Presently I am running CentOS release 5.5 (Final). I am looking to setup bridging as I would like to setup some KVM virtual hosts on my system as a
Time to test if ping works:
~ $ ping -c3 192.168.1.254 PING 192.168.1.254 (192.168.1.254) 56(84) bytes of data. ping: sendmsg: Operation not permitted ping: sendmsg: Operation not permitted ping: sendmsg: Operation not permitted
Did you remember to brctl addif the regular interfaces?
Nope, that is what I had forgotten. Thnx
On Fri, Nov 26, 2010 at 9:15 AM, Robert Spangler mlists@zoominternet.net wrote:
Hello,
Presently I am running CentOS release 5.5 (Final). I am looking to setup bridging as I would like to setup some KVM virtual hosts on my system as a test lab. I am following the the instruction at this site
(snip)
Everything is back to normal. I cannot figure out what am I missing here? Interfaces and routing look to be setup correctly. Is there something else I need to be looking at?
I recommend you look at the documentaion available from docs.redhat.com. For setting up bridged networking, see:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html-single/Vir...
Akemi
On Friday 26 November 2010 12:27, Akemi Yagi wrote:
I recommend you look at the documentaion available from docs.redhat.com. For setting up bridged networking, see:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html-single/Vi rtualization/index.html#sect-Virtualization-Network_Configuration-Bridged_ne tworking_with_libvirt
Thank you kindly for the link. I have some reading a head of me.
On Fri, Nov 26, 2010 at 02:12:04PM -0500, Robert Spangler wrote:
On Friday 26 November 2010 12:27, Akemi Yagi wrote:
I recommend you look at the documentaion available from docs.redhat.com. For setting up bridged networking, see:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html-single/Vi rtualization/index.html#sect-Virtualization-Network_Configuration-Bridged_ne tworking_with_libvirt
Thank you kindly for the link. I have some reading a head of me.
Much as I respect Akemi san, I would say, don't bother. It's the usual poorly written RH documentation.
In contrast, there is the CentOS wiki, written by someone who actually knows something about writing documentation that people can understand. Oh....wait, it's me.
Actually, the KVM wiki article is very out of date, but the section on bridging is applicable. The RH docs were so bad, that I still had to go back to my own article.
http://wiki.centos.org/HowTos/KVM
On Fri, Nov 26, 2010 at 5:16 PM, Scott Robbins scottro@nyc.rr.com wrote:
On Fri, Nov 26, 2010 at 02:12:04PM -0500, Robert Spangler wrote:
On Friday 26 November 2010 12:27, Akemi Yagi wrote:
I recommend you look at the documentaion available from docs.redhat.com. For setting up bridged networking, see:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5/html-single/Vir...
Much as I respect Akemi san, I would say, don't bother. It's the usual poorly written RH documentation.
In contrast, there is the CentOS wiki, written by someone who actually knows something about writing documentation that people can understand. Oh....wait, it's me.
Actually, the KVM wiki article is very out of date, but the section on bridging is applicable. The RH docs were so bad, that I still had to go back to my own article.
Mmm? I may not be the biggest fan of the Red Hat docs but I have to give a good score to that one about bridged networking. I followed the instructions in there and had no problem setting it up on my KVM hosts.
It has couple of points the OP may need to know. One is that NetworkManager needs to be disabled. The other is how to handle iptables (OP disable it while troubleshooting).
Akemi
On Fri, Nov 26, 2010 at 06:09:26PM -0800, Akemi Yagi wrote:
On Fri, Nov 26, 2010 at 5:16 PM, Scott Robbins scottro@nyc.rr.com wrote:
Mmm? I may not be the biggest fan of the Red Hat docs but I have to give a good score to that one about bridged networking. I followed the instructions in there and had no problem setting it up on my KVM hosts.
It has couple of points the OP may need to know. One is that NetworkManager needs to be disabled. The other is how to handle iptables (OP disable it while troubleshooting).
Ah, aikawarazu, good point. Not using NetworkManager--to be honest, I find it causes more problems than it solves, I was't aware of that.
(The wiki article does mention additions to iptables.)
Regrettably, however, I've found KVM to be somewhat of a disappointment. (My own personal experience.) For any workstation, I'm finding VMware-player to be the new contender, running guests faster than the later VirtualBoxes and/or KVM, and for serious production, I
On Friday 26 November 2010 21:47, Scott Robbins wrote:
It has couple of points the OP may need to know. One is that NetworkManager needs to be disabled. The other is how to handle iptables (OP disable it while troubleshooting).
Ah, aikawarazu, good point. Not using NetworkManager--to be honest, I find it causes more problems than it solves, I was't aware of that.
Nor do I. I prefer to configure my system for the CLI.
(The wiki article does mention additions to iptables.)
Nice. Will have to take a look at this one too.
Oh, thnx for your input.
At Fri, 26 Nov 2010 12:15:51 -0500 CentOS mailing list centos@centos.org wrote:
Hello,
Presently I am running CentOS release 5.5 (Final). I am looking to setup bridging as I would like to setup some KVM virtual hosts on my system as a test lab. I am following the the instruction at this site
but I cannot figure out where I am going wrong and would be thankful if someone could point me in the right direction.
Here is what I have done:
Check bridge information with the following:
~ $ modprobe -v bridge
No issues or errors
~ $ cat /proc/modules | grep bridge bidge 91889 0 - Live 0xffffffff89247000
Check to ensure forwarding is turned on:
~ $ cat /proc/sys/net/ipv4/ip_forward 1
Checked that my interface are up and running (Was sure of this but did the check anyway):
~ $ ifconfig eth0 Link encap:Ethernet HWaddr 48:5B:39:2A:07:D5 inet addr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::4a5b:39ff:fe2a:7d5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1059 errors:0 dropped:0 overruns:0 frame:0 TX packets:1080 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:454226 (443.5 KiB) TX bytes:120584 (117.7 KiB) Interrupt:90 Base address:0x8400
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:92 errors:0 dropped:0 overruns:0 frame:0 TX packets:92 errors:0 dropped:0 overruns:0 carrier:0[Thu Nov 25
So now I begin to create the bridge form CLI as I want to make sure everything works before committing it to the config:
brctl addbr br0 ifconfig eth0 down ifconfig br0 192.168.1.100 up ifconfig eth0 0.0.0.0 up
brctl addif br0 eth0
You need to add the physical interface(s) to the bridge interface.
You can set this up to go automagically like this:
sauron.deepsoft.com% cat /etc/sysconfig/network-scripts/ifcfg-eth0 # nVidia Corporation MCP77 Ethernet DEVICE=eth0 BOOTPROTO=static HWADDR=00:19:66:D6:ED:93 ONBOOT=yes BRIDGE=br0
sauron.deepsoft.com% cat /etc/sysconfig/network-scripts/ifcfg-br0 DEVICE=br0 TYPE=Bridge BOOTPROTO=static BROADCAST=192.168.250.255 IPADDR=192.168.250.1 NETMASK=255.255.255.0 NETWORK=192.168.250.0 ONBOOT=yes
(change as needed to match your interaces and ipaddresses, etc.)
route add default gw 192.168.1.254
I check my interfaces and routing:
~ $ ifconfig br0 Link encap:Ethernet HWaddr 00:00:00:00:00:00 inet addr:192.168.1.100 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::200:ff:fe00:0/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:5 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 b) TX bytes:398 (398.0 b)
eth0 Link encap:Ethernet HWaddr 48:5B:39:2A:07:D5 inet6 addr: fe80::4a5b:39ff:fe2a:7d5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:64662 errors:0 dropped:0 overruns:0 frame:0 TX packets:63301 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:17699194 (16.8 MiB) TX bytes:7958063 (7.5 MiB) Interrupt:90 Base address:0x8400
lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:211 errors:0 dropped:0 overruns:0 frame:0 TX packets:211 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:17346 (16.9 KiB) TX bytes:17346 (16.9 KiB)
~ $ route -n Kernel IP routing table Destination Gateway Genmask Flags Metric Ref Use
Iface
192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0 0.0.0.0 192.168.1.254 0.0.0.0 UG 0 0 0 br0
Time to test if ping works:
~ $ ping -c3 192.168.1.254 PING 192.168.1.254 (192.168.1.254) 56(84) bytes of data. ping: sendmsg: Operation not permitted ping: sendmsg: Operation not permitted ping: sendmsg: Operation not permitted
--- 192.168.1.254 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2000ms
I know the firewall is causing this issue so I stop the firewall:
~ $ service iptables stop Flushing firewall rules: [ OK ] Setting chains to policy ACCEPT: nat filter [ OK ] Unloading iptables modules: [ OK ]
Time to try ping again:
~ $ ping -c3 192.168.1.254 PING 192.168.1.254 (192.168.1.254) 56(84) bytes of data. From 192.168.1.100 icmp_seq=1 Destination Host Unreachable From 192.168.1.100 icmp_seq=2 Destination Host Unreachable From 192.168.1.100 icmp_seq=3 Destination Host Unreachable
--- 192.168.1.254 ping statistics --- 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 1999ms
To back out all I needed to do was:
ifconfig br0 down brctl delbr br0 service iptables start service netowrk restart
Everything is back to normal. I cannot figure out what am I missing here? Interfaces and routing look to be setup correctly. Is there something else I need to be looking at?
On Friday 26 November 2010 12:28, Robert Heller wrote:
works before committing it to the config:
brctl addbr br0 ifconfig eth0 down ifconfig br0 192.168.1.100 up ifconfig eth0 0.0.0.0 up
brctl addif br0 eth0
You need to add the physical interface(s) to the bridge interface.
Yes, thank you for this information. This is the set I had missed.
You can set this up to go automagically like this:
sauron.deepsoft.com% cat /etc/sysconfig/network-scripts/ifcfg-eth0 # nVidia Corporation MCP77 Ethernet DEVICE=eth0 BOOTPROTO=static HWADDR=00:19:66:D6:ED:93 ONBOOT=yes BRIDGE=br0
sauron.deepsoft.com% cat /etc/sysconfig/network-scripts/ifcfg-br0 DEVICE=br0 TYPE=Bridge BOOTPROTO=static BROADCAST=192.168.250.255 IPADDR=192.168.250.1 NETMASK=255.255.255.0 NETWORK=192.168.250.0 ONBOOT=yes
Thnx again for this information.
On Fri, Nov 26, 2010 at 12:15 PM, Robert Spangler mlists@zoominternet.net wrote:
Hello,
Presently I am running CentOS release 5.5 (Final). I am looking to setup bridging as I would like to setup some KVM virtual hosts on my system as a test lab. I am following the the instruction at this site
Don't bother.
The Qemu based tools in libvirt, and their implementation in virt-manager, should be taken out back and forced to read Eric Raymond's screed on open source interfaces (The Luxury of Ignorance). Compatibility with arbitrary virtualization suites is not your friend when it's done that badly. Simple operations, like "set up two disks at first setup", are not possible from the GUI. This is one among numerous utilities available from the command line setup tool that are not accessible from the GUI: that's just a failure of GUI design.
KVM, itself, was unusable in my testing due to the "bridged network" mishandling and its complete lack of a concept of failover for network issues, particularly pair bonding for the server itself. PXE for the clients was unusable, and it ran like a dyslexic on too many opiates, slow, twitchy, and unpredicatable.
VMWare works well, even the free personal versions, and Xen used to work well (although its purchase by Citrix has me concerned, I've not played with it in 2 years now, and I'm very unhappy with libvirt.)
On November 26, 2010 11:25:06 am Nico Kadel-Garcia wrote:
KVM, itself, was unusable in my testing due to the "bridged network" mishandling and its complete lack of a concept of failover for network issues, particularly pair bonding for the server itself. PXE for the clients was unusable, and it ran like a dyslexic on too many opiates, slow, twitchy, and unpredicatable.
The UI in RHEL 5/CentOS 5 is definitely very limited, but KVM does work with all these things under the hood. In particular KVM seems to run fine on top of a simple host bridge, which can in turn rely on a bonded interface. I have had no problems with boot support, although I confess I don't use PXE - DHCP and kickstart over the LAN work fine, though.
I do hope the interface implementation in RHEL 6 will be much more usable. I don't mind doing things in XML files and command lines, but lots of people do.
On Fri, Nov 26, 2010 at 3:00 PM, Alan Hodgson ahodgson@simkin.ca wrote:
On November 26, 2010 11:25:06 am Nico Kadel-Garcia wrote:
KVM, itself, was unusable in my testing due to the "bridged network" mishandling and its complete lack of a concept of failover for network issues, particularly pair bonding for the server itself. PXE for the clients was unusable, and it ran like a dyslexic on too many opiates, slow, twitchy, and unpredicatable.
The UI in RHEL 5/CentOS 5 is definitely very limited, but KVM does work with all these things under the hood. In particular KVM seems to run fine on top of a simple host bridge, which can in turn rely on a bonded interface. I have had no problems with boot support, although I confess I don't use PXE - DHCP and kickstart over the LAN work fine, though.
It is not merely "limited".
PXE is very common for server installations of brand new hardware, or for remote KVM managed hardware, to avoid having to pop a CD in it. It's well undertood, and I got nowhere, even with it for KVM. (VMWare and Xen worked fine.)
On Sat, Nov 27, 2010 at 6:23 AM, Nico Kadel-Garcia nkadel@gmail.com wrote:
On Fri, Nov 26, 2010 at 3:00 PM, Alan Hodgson ahodgson@simkin.ca wrote:
On November 26, 2010 11:25:06 am Nico Kadel-Garcia wrote:
KVM, itself, was unusable in my testing due to the "bridged network" mishandling and its complete lack of a concept of failover for network issues, particularly pair bonding for the server itself. PXE for the clients was unusable, and it ran like a dyslexic on too many opiates, slow, twitchy, and unpredicatable.
The UI in RHEL 5/CentOS 5 is definitely very limited, but KVM does work with all these things under the hood. In particular KVM seems to run fine on top of a simple host bridge, which can in turn rely on a bonded interface. I have had no problems with boot support, although I confess I don't use PXE - DHCP and kickstart over the LAN work fine, though.
It is not merely "limited".
PXE is very common for server installations of brand new hardware, or for remote KVM managed hardware, to avoid having to pop a CD in it. It's well undertood, and I got nowhere, even with it for KVM. (VMWare and Xen worked fine.)
interesting. I have a working home lab with KVM and I bootstrap all my vm's from pxe, both win and lin. So I know it works fine. Not managed from the virtual machine manager, though. Next year I will be evaluating it, and it has better support pxe :)
-
Akemi Yagi -
Alan Hodgson -
Eduardo Grosclaude -
Natxo Asenjo -
Nico Kadel-Garcia -
Robert Heller -
Robert Spangler -
Scott Robbins