[CentOS-virt] Finally switching from Xen to KVM - question about networking

Wed Jun 11 14:03:28 UTC 2014
Zoltan Frombach <zoltan at frombach.com>

On 6/11/2014 3:50 PM, Steve Campbell wrote:
> On 6/10/2014 4:00 PM, Zoltan Frombach wrote:
>> On 6/10/2014 9:51 PM, Steve Campbell wrote:
>>> On 6/10/2014 3:38 PM, Zoltan Frombach wrote:
>>>> On 6/10/2014 9:27 PM, Steve Campbell wrote:
>>>>> On 6/10/2014 3:09 PM, Zoltan Frombach wrote:
>>>>>> Steve,
>>>>>>
>>>>>> Try the following config.
>>>>>>
>>>>>> On your host:
>>>>>>
>>>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>>>
>>>>>> DEVICE=eth0
>>>>>> BOOTPROTO=none
>>>>>> ONBOOT=yes
>>>>>> NM_CONTROLLED=no
>>>>>> TYPE=Ethernet
>>>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- put your physical NIC's MAC address
>>>>>> here
>>>>>> BRIDGE=br0
>>>>>> USERCTL=no
>>>>>> IPV6INIT=no
>>>>>> IPV6_AUTOCONF=no
>>>>>>
>>>>>>
>>>>>> /etc/sysconfig/network-scripts/ifcfg-br0    :
>>>>>>
>>>>>> DEVICE=br0
>>>>>> BOOTPROTO=static
>>>>>> ONBOOT=yes
>>>>>> NM_CONTROLLED=no
>>>>>> TYPE=Bridge
>>>>>> IPADDR=10.0.5.16
>>>>>> NETMASK=255.255.255.0   <- you may need to adjust this to your
>>>>>> network
>>>>>> GATEWAY=10.0.5.1
>>>>>> DNS1=8.8.8.8
>>>>>> DNS2=8.8.4.4
>>>>>> PEERDNS=yes
>>>>>> DELAY=0
>>>>>> STP=off
>>>>>> USERCTL=no
>>>>>> IPV6INIT=no
>>>>>> IPV6_AUTOCONF=no
>>>>>>
>>>>>>
>>>>>> Restart networking on the host.
>>>>>>
>>>>>> Then inside you VM:
>>>>>>
>>>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>>>
>>>>>> DEVICE=eth0
>>>>>> BOOTPROTO=static
>>>>>> ONBOOT=yes
>>>>>> NM_CONTROLLED=no
>>>>>> TYPE=Ethernet
>>>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- it must be a unique MAC address for
>>>>>> your VM
>>>>>> IPADDR=10.0.5.17
>>>>>> NETMASK=255.255.255.0     <- you may need to adjust this to your
>>>>>> network
>>>>>> GATEWAY=10.0.5.1
>>>>>> DNS1=8.8.8.8
>>>>>> DNS2=8.8.4.4
>>>>>> PEERDNS=yes
>>>>>> USERCTL=no
>>>>>> IPV6INIT=no
>>>>>> IPV6_AUTOCONF=no
>>>>>>
>>>>>>
>>>>>> Then in Virtual Machine Manager make sure that your VM's NIC is
>>>>>> connected to the br0 bridge, like this:
>>>>>>
>>>>>> Network Source: Specify shared device name
>>>>>>          Bridge Name: br0
>>>>>>
>>>>>>
>>>>>>
>>>>>> On 6/10/2014 8:16 PM, Steve Campbell wrote:
>>>>>>> On 6/10/2014 12:43 PM, Digimer wrote:
>>>>>>>> On 10/06/14 12:38 PM, Steve Campbell wrote:
>>>>>>>>> On 6/10/2014 12:05 PM, Digimer wrote:
>>>>>>>>>> On 10/06/14 11:46 AM, Steve Campbell wrote:
>>>>>>>>>>> On 6/10/2014 10:46 AM, Digimer wrote:
>>>>>>>>>>>> On 10/06/14 10:03 AM, Steve Campbell wrote:
>>>>>>>>>>>>> I had so much trouble putting Centos 6 guest VMs on a Centos 5
>>>>>>>>>>>>> host
>>>>>>>>>>>>> that
>>>>>>>>>>>>> I finally switched to a Centos 6 host.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I've not needed more that test VMs, so I've used Virtual
>>>>>>>>>>>>> Machine
>>>>>>>>>>>>> Manager
>>>>>>>>>>>>> on the old system, which worked pretty well, so I decided to
>>>>>>>>>>>>> create my
>>>>>>>>>>>>> first KVM guest machine. I noticed when I created it, I only
>>>>>>>>>>>>> had the
>>>>>>>>>>>>> options of NAT for my network interface, so I used that
>>>>>>>>>>>>> (obvious).
>>>>>>>>>>>>>
>>>>>>>>>>>>> Well, after starting the VM, I find I don't have connectivity
>>>>>>>>>>>>> with
>>>>>>>>>>>>> that
>>>>>>>>>>>>> interface. Reading, I find examples where I need to create
>>>>>>>>>>>>> bridges
>>>>>>>>>>>>> perhaps. Xen did most of this for me, so it's a little new to
>>>>>>>>>>>>> me.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Can anyone throw me a clue, please?
>>>>>>>>>>>>>
>>>>>>>>>>>>> steve campbell
>>>>>>>>>>>> Setting up a bridge is not that hard, and it will give your VMs
>>>>>>>>>>>> direct
>>>>>>>>>>>> access to the outside world, and host <-> VM access just
>>>>>>>>>>>> fine as
>>>>>>>>>>>> well.
>>>>>>>>>>>>
>>>>>>>>>>>> Here is a link showing how to setup a bridge connected to a
>>>>>>>>>>>> bond
>>>>>>>>>>>> device. Ignore the bond and pretend it is a straight ethX
>>>>>>>>>>>> device:
>>>>>>>>>>>>
>>>>>>>>>>>> https://alteeve.ca/w/AN!Cluster_Tutorial_2#Configuring_our_Bridge.2C_Bonds_and_Interfaces
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> The host has a device named virbr0 that is installed during
>>>>>>>>>>> system
>>>>>>>>>>> installation. It also has a network device vnet0. There are no
>>>>>>>>>>> files in
>>>>>>>>>>> /etc/sysconfig/network-scripts for these.
>>>>>>>>>>>
>>>>>>>>>>> Shouldn't I be able to use the virbr0 virtual bridge for this?
>>>>>>>>>>> I've
>>>>>>>>>>> tried setting up the VM's device with all of the options that is
>>>>>>>>>>> listed,
>>>>>>>>>>> but to no avail.
>>>>>>>>>>>
>>>>>>>>>>> Should I need to set up another bridge for this?
>>>>>>>>>>>
>>>>>>>>>>> And thanks for the link.
>>>>>>>>>>>
>>>>>>>>>>> steve
>>>>>>>>>> virbr0 is created and managed by libvirtd. If you open Virtual
>>>>>>>>>> Machine
>>>>>>>>>> Manager, connect to localhost and then double-click on
>>>>>>>>>> 'localhost',
>>>>>>>>>> you will see a tab for creating/managing bridges (NAT'ed,
>>>>>>>>>> generally).
>>>>>>>>>> I disable 'virbr0' as NAT'ing is generally not what I want.
>>>>>>>>>>
>>>>>>>>>> The 'vnetX' devices are dynamically created to link a VM's
>>>>>>>>>> interface
>>>>>>>>>> to a bridge. Think of them as virtual network cables. They get
>>>>>>>>>> created
>>>>>>>>>> and destroyed as needed.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Sorry, but I'm confused:
>>>>>>>>>
>>>>>>>>> My host server has a real NIC and IP address with a real
>>>>>>>>> gateway to
>>>>>>>>> the
>>>>>>>>> outside:
>>>>>>>>> virtbr0 IP: 192.168.122.1
>>>>>>>>> Host IP: 10.0.5.16
>>>>>>>>> Gateway IP: 10.0.5.1
>>>>>>>>> on eth0 and this works
>>>>>>>>>
>>>>>>>>> My VM server has all fake stuff currently:
>>>>>>>>> Host IP: 10.0.5.17
>>>>>>>>> Gateway IP: 10.0.5.1
>>>>>>>>> on eth0 and this is like NIC without a cable.
>>>>>>>>>
>>>>>>>>> So I need to create a bridge device on both the host and VM (lets
>>>>>>>>> say I
>>>>>>>>> name it br1). I change the eth0 config file on both host and VM to
>>>>>>>>> point
>>>>>>>>> to br1 and give the br1 config file on both host and VM the
>>>>>>>>> correct
>>>>>>>>> IP.
>>>>>>>>> But won't this just let the two talk to each other. How will
>>>>>>>>> the VM
>>>>>>>>> server get outside?
>>>>>>>>>
>>>>>>>>> steve
>>>>>>>> The bridge is created only on the host. Think of the bridge as
>>>>>>>> being
>>>>>>>> like a virtual switch. When a VM is created, you tell it to
>>>>>>>> connect to
>>>>>>>> the bridge, similar to how you would plug a physical wire into a
>>>>>>>> real
>>>>>>>> switch. That provides the link to the network, and then you
>>>>>>>> configure
>>>>>>>> the virtual server's network just as if it was a real network.
>>>>>>>>
>>>>>>>> On the host, you don't set the IP on the ethX device, instead you
>>>>>>>> tell
>>>>>>>> ethX to connect to the bridge (look at 'bond2' in the link above
>>>>>>>> for
>>>>>>>> the 'BRIDGE="..."' line). Then you move the host's IP
>>>>>>>> address/config
>>>>>>>> to the bridge itself (look at the 'vbr2' in the link above).
>>>>>>>>
>>>>>>>> If you're still stuck, tell me the IP addresses you want to set in
>>>>>>>> the
>>>>>>>> host and VM and I'll see if I can bang out a couple sample
>>>>>>>> 'ifcfg-X'
>>>>>>>> files.
>>>>>>>>
>>>>>>> I'm running into some problems with my config files where I get a
>>>>>>> message that says the network can't determine my interface names.
>>>>>>> I'll
>>>>>>> look further, but the IPs above are what I need. 10.0.5.1 is a
>>>>>>> firewall
>>>>>>> that NATs the 10.0.5.0/24 network to another outside public
>>>>>>> firewall.
>>>>>>>
>>>>>>> Thanks
>>>>>>> steve
>>>>>>> _______________________________________________
>>>>>>> CentOS-virt mailing list
>>>>>>> CentOS-virt at centos.org
>>>>>>> http://lists.centos.org/mailman/listinfo/centos-virt
>>>>>>>
>>>>> Stuck in a rut now. Restarting the network indicates :
>>>>>
>>>>> Device eth0 does not seem to be present, delaying initialization.
>>>>>
>>>>> I thought it might be due to the fact I didn't have the "TYPE" in the
>>>>> ifcfg-eth0 file, but after adding that, I still get the error.
>>>> NM_CONTROLLED=no might help with that.
>>> I turned off and stopped NetworkManager and had already specified this.
>>>>> I'm using the same hardware address that was in the original eth0
>>>>> file.
>>>> That is fine.
>>>>
>>>>>     My bridge device starts fine, but I'll tailor it to your specs.
>>>> Also, you might want to check the contents of your
>>>> /etc/udev/rules.d/70-persistent-net.rules file. Is the MAC address for
>>>> eth0 the same as in your /etc/sysconfig/network-scripts/ifcfg-eth0
>>>> file?
>>>>
>>> They are the same except the udev file has lower case letters in it.
>> You mean the hexadecimal numbers are lower case? That shouldn't matter.
>>
>> Try to delete the file /etc/udev/rules.d/70-persistent-net.rules and
>> reboot your host machine.
>> ( See
>> http://www.andrewklau.com/device-eth0-does-not-seem-to-be-present-delaying-initialization/
>> )
>>
> All is well on the host now, and I've brought up the VM and configured
> it to use the bridge. I can ping the host from the VM but I have no
> access beyond the local network. The host sees the world, but the VM
> only sees the 10.0.5.0/24 network.
Did you set the gateway in your VM to 10.0.5.1 ?
Can you ping the gateway ( ping 10.0.5.1 ) from the VM?
Do you have a firewall configured in your VM (or host) ? If so, try to 
disable the firewall temporarily for testing.