took a couple months off due to road blocks, hoping a fresh look would allow me to use my server as desired.
I have three ports, eth0-2 over two nics. I want to bond them, and then use a bridge to connect to virtual machines on the virtual host computer. Never having done this, I am confused on some parts.
Here is where I am at now and any pointers helpful. Single server, centos 6.x My issue is do I add the ip address of the virtual host and where? Do I not add any of the ip addresses of the virtual machines? (thus they will be looked for via the bridge?) Each virtual machine will be getting its own ip address as it will be used for a website.
here is the basic configuration I am looking at, with some things not added like hwaddress and a few tweaks (zeroconf?)
am I even close? I admit I have never done this and have no mentor other than this exceptional mailing list.
============================ /etc/modprobe.conf alias bond0 bonding options bond0 mode=balance-alb miimon=100 ============================
============================ bond0 DEVICE=bond0 BOOTPROTO=none BRIDGE=br0 ONBOOT=yes BONDING_OPTS="mode=1 miimon=100 primary=eth0" ============================
============================ br0 DEVICE=br0 TYPE=Bridge BOOTPROTO=none IPADDR=<my virtual host IP address?> NETWORK=xxx.xx.xx.xxx BROADCAST=xxx.xx.xx.xx NETMASK=255.255.255.0 ONBOOT=yes GATEWAY=xx.xxx.xxx.xx ============================
I did not add the hwaddress to the items below. ======================================== eth0 DEVICE=eth0 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none
eth1 DEVICE=eth0 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none
eth2 DEVICE=eth0 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none =========================================
I do this in my clusters (using six NICs, but that doesn't matter). I've documented how I create bonds and bridges with details here:
https://alteeve.com/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Network
Scroll down a bit to:
https://alteeve.com/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Configuring_The_Br...
and
https://alteeve.com/w/2-Node_Red_Hat_KVM_Cluster_Tutorial#Network
In short; - Put the host's IP on the bridge, not the bond. - HWADDR are defined in the interfaces only, not the bond or bridge.
Digimer
On 01/27/2012 05:16 PM, Bob Hoffman wrote:
took a couple months off due to road blocks, hoping a fresh look would allow me to use my server as desired.
I have three ports, eth0-2 over two nics. I want to bond them, and then use a bridge to connect to virtual machines on the virtual host computer. Never having done this, I am confused on some parts.
Here is where I am at now and any pointers helpful. Single server, centos 6.x My issue is do I add the ip address of the virtual host and where? Do I not add any of the ip addresses of the virtual machines? (thus they will be looked for via the bridge?) Each virtual machine will be getting its own ip address as it will be used for a website.
here is the basic configuration I am looking at, with some things not added like hwaddress and a few tweaks (zeroconf?)
am I even close? I admit I have never done this and have no mentor other than this exceptional mailing list.
============================ /etc/modprobe.conf alias bond0 bonding options bond0 mode=balance-alb miimon=100 ============================
============================ bond0 DEVICE=bond0 BOOTPROTO=none BRIDGE=br0 ONBOOT=yes BONDING_OPTS="mode=1 miimon=100 primary=eth0" ============================
============================ br0 DEVICE=br0 TYPE=Bridge BOOTPROTO=none IPADDR=<my virtual host IP address?> NETWORK=xxx.xx.xx.xxx BROADCAST=xxx.xx.xx.xx NETMASK=255.255.255.0 ONBOOT=yes GATEWAY=xx.xxx.xxx.xx ============================
I did not add the hwaddress to the items below.
eth0 DEVICE=eth0 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none
eth1 DEVICE=eth0 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none
eth2 DEVICE=eth0 USERCTL=no ONBOOT=yes MASTER=bond0 SLAVE=yes BOOTPROTO=none =========================================
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
well, I get no errors on network restart, but all that shows starting up is the loopback. ifconfig shows br0 with all settings right.
here is what I have changed, still cannot access outside world with the virtual host machine. I am sure it is a resolution thing, but the network restart does not mention the bridge. No copy paste as using an ipmi card, so all hand typed here, so the capitalization is not correct in this post.
Since this is a virtual host only, I did not do 'bind' and am just using google dns servers until I set up my own for resolution.
netstat -rn shows two Br0 1 has the destination ip and genamsk, the other has the gateway
/etc/hosts
xx.xx.xx.xx main.xxx.com main 127.0.0.1 localhost.locadomain localhost ::1 main.xxx.com main localhost6.localdomain6 localhost6
/etc/host.conf order hosts,bind
/etc/resolv.conf search main.xxx.com nameserver=8.8.8.8 nameserver=8.8.4.4
/etc/sysconfig/iptables (this is the basic install, nothing was changed by me) (copied by hand, did not type capital letters)
:input accept [0:0] :forward accept [0:0] :output accept [0:0] -A input -m --state established,related -j accept -a input -p icmp -j accept -a input -i lo -j accept -a input -m state --state NEW -m tcp -p tcp --dport 22 -j accept -i forward -m physdev --physdev-is-bridged -j accept -a imput -j reject --reject-with icmp-host-prohibited commit
/etc/sysconfig/network NETOWRKING=yes HOSTNAME=main.xxx.com
/etc//modprobe.d/bonding.conf alias bond0 bonding
/etc/sysconfig/network-scripts/br0 (this file is where I typed in the ip of my machine.)
(normal stuff) Device=br0 type=BRIDGE nootproto=static ipaddr, net, broadcast, netmask, etc.. as should be nozeroconf=yes onboot yes defroute yes
bond0
device bond0 nm controlled no bootproto no beridge br0 bonding opts, etc..
eths are all master bond0 slave yes, nm controlled no.
One thing at a time, let's get the bond working, then come back and add the bridge.
They should look something like this;
# Back-Channel Network - Link 1 HWADDR="00:E0:81:C7:EC:49" DEVICE="eth0" NM_CONTROLLED="no" ONBOOT="yes" BOOTPROTO="none" MASTER="bond0" SLAVE="yes"
and
# Back-Channel Network - Link 2 HWADDR="00:1B:21:9D:59:FC" DEVICE="eth3" NM_CONTROLLED="no" ONBOOT="yes" BOOTPROTO="none" MASTER="bond0" SLAVE="yes"
Then the bond config should look like this:
# Back-Channel Network - Bond DEVICE="bond0" BOOTPROTO="static" NM_CONTROLLED="no" ONBOOT="yes" BONDING_OPTS="mode=1 miimon=100 use_carrier=1 updelay=120000 downdelay=0 primary=eth0" IPADDR="10.20.0.1" NETMASK="255.255.0.0"
Now try to restart the network and see if the bond and it's interfaces come up. If so, then we'll come back to the bridge. One thing I can say right now though is that the name of the bridge has to sort *after* the bond interfaces. For this reason, I always recommend 'vbrX' for bridge names.
On 01/27/2012 07:32 PM, Bob Hoffman wrote:
well, I get no errors on network restart, but all that shows starting up is the loopback. ifconfig shows br0 with all settings right.
here is what I have changed, still cannot access outside world with the virtual host machine. I am sure it is a resolution thing, but the network restart does not mention the bridge. No copy paste as using an ipmi card, so all hand typed here, so the capitalization is not correct in this post.
Since this is a virtual host only, I did not do 'bind' and am just using google dns servers until I set up my own for resolution.
netstat -rn shows two Br0 1 has the destination ip and genamsk, the other has the gateway
/etc/hosts
xx.xx.xx.xx main.xxx.com main 127.0.0.1 localhost.locadomain localhost ::1 main.xxx.com main localhost6.localdomain6 localhost6
/etc/host.conf order hosts,bind
/etc/resolv.conf search main.xxx.com nameserver=8.8.8.8 nameserver=8.8.4.4
/etc/sysconfig/iptables (this is the basic install, nothing was changed by me) (copied by hand, did not type capital letters)
:input accept [0:0] :forward accept [0:0] :output accept [0:0] -A input -m --state established,related -j accept -a input -p icmp -j accept -a input -i lo -j accept -a input -m state --state NEW -m tcp -p tcp --dport 22 -j accept -i forward -m physdev --physdev-is-bridged -j accept -a imput -j reject --reject-with icmp-host-prohibited commit
/etc/sysconfig/network NETOWRKING=yes HOSTNAME=main.xxx.com
/etc//modprobe.d/bonding.conf alias bond0 bonding
/etc/sysconfig/network-scripts/br0 (this file is where I typed in the ip of my machine.)
(normal stuff) Device=br0 type=BRIDGE nootproto=static ipaddr, net, broadcast, netmask, etc.. as should be nozeroconf=yes onboot yes defroute yes
bond0
device bond0 nm controlled no bootproto no beridge br0 bonding opts, etc..
eths are all master bond0 slave yes, nm controlled no.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
well...still cleaning up files to find what is and is not needed in those eth, bond, and br files...
here goes..
one.... eth files nm_controlled no, master=bond0, slave=yes no ipaddresses or any of that stuff...
bond file.. #1 thing....it must be name ifcfg-bond0, not just bond0 However, the eth files must reference solely as bond0 without the ifcfg. the bond file, for the moment had the ip addresses and stuff, not sure if I need them there or not. (probably do not need them, but it worked with them) nm_controlled=no BRIDGE=br0 bonding options go in this file too.
in etc/modprobe.d/ you need to make a file called bonding.conf add this line alias bond0 bonding
the ifcfg-br0 file, MUST be named ifcfg-br0 (or whatever your bridge is name, br1, br2, etc) should contain type=Bridge VERY IMPORTANT YOU SPELL IT WITH A CAPITAL B, and lowercase the rest of the word bridge. (that is what was needed...go figure) add the ipaddresses, netmask, etc etc etc to this file
resolv.conf you want to add your dns servers, or for a quickie fix add nameserver 8.8.8.8 nameserver 8.8.4.4 (these are googles free dns resolvers)
anyway, this is just a quickie solution, will be posting a vid of each step soon.. finally!!!!!!!!!!!
updated from 6.0 to the new version....big update...gotta wait...won't let me access the VMs while updating...sounds scary...yikes
On 01/27/2012 10:03 PM, Bob Hoffman wrote:
well...still cleaning up files to find what is and is not needed in those eth, bond, and br files...
here goes..
one.... eth files nm_controlled no, master=bond0, slave=yes no ipaddresses or any of that stuff...
Right.
bond file.. #1 thing....it must be name ifcfg-bond0, not just bond0
To clarify, the *file* needs to be 'ifcfg-bondX', not the device name, which is DEVICE="bondX". The file name is used to find which files to examine/start when network starts. The later is used when interacting with the device itself (ie: ifup bond0).
However, the eth files must reference solely as bond0 without the ifcfg. the bond file, for the moment had the ip addresses and stuff, not sure if I need them there or not.
Exactly. The file is 'ifcfg-X', the device itself is just 'X'.
(probably do not need them, but it worked with them) nm_controlled=no BRIDGE=br0 bonding options go in this file too.
Yes, but remember to change the bond file name to 'ifcfg-vbrX'. It's ideal to have the device name match the file name, minus the 'ifcfg-' prefix, but it is not required.
in etc/modprobe.d/ you need to make a file called bonding.conf add this line alias bond0 bonding
I don't use this on EL6, not sure if it's really required anymore.
the ifcfg-br0 file, MUST be named ifcfg-br0 (or whatever your bridge is name, br1, br2, etc)
Again, the *file* name *must* sort after the bond file, which 'ifcfg-br0' does *not*. The reason is that the bond interface has to be up before the bridge.
should contain type=Bridge VERY IMPORTANT YOU SPELL IT WITH A CAPITAL B, and lowercase the rest of the word bridge.
Yes, this is an easy to miss point.
(that is what was needed...go figure) add the ipaddresses, netmask, etc etc etc to this file
resolv.conf you want to add your dns servers, or for a quickie fix add nameserver 8.8.8.8 nameserver 8.8.4.4 (these are googles free dns resolvers)
You can, alternatively, define the DNS in the bridge file (or really, any device config where you specify an IP address). I like to do this as it puts all the stuff I need to manage into one file. For example;
vim /etc/sysconfig/network-scripts/ifcfg-vbr2
# Internet-Facing Network - Bridge DEVICE="vbr2" TYPE="Bridge" BOOTPROTO="static" IPADDR="10.255.0.1" NETMASK="255.255.0.0" GATEWAY="10.255.255.254" DNS1="192.139.81.117" DNS2="192.139.81.1" DEFROUTE="yes"
anyway, this is just a quickie solution, will be posting a vid of each step soon.. finally!!!!!!!!!!!
updated from 6.0 to the new version....big update...gotta wait...won't let me access the VMs while updating...sounds scary...yikes
The update of EL6 to a new y-stream release should *not* interfere with access to VMs. I suspect something else is causing a problem, like running out of bandwidth or similar.
-
Bob Hoffman -
Digimer