[CentOS-virt] Finally switching from Xen to KVM - question about networking

Wed Jun 11 14:10:34 UTC 2014
Steve Campbell <campbell at cnpapers.com>

On 6/11/2014 10:03 AM, Zoltan Frombach wrote:
>
> On 6/11/2014 3:50 PM, Steve Campbell wrote:
>> On 6/10/2014 4:00 PM, Zoltan Frombach wrote:
>>> On 6/10/2014 9:51 PM, Steve Campbell wrote:
>>>> On 6/10/2014 3:38 PM, Zoltan Frombach wrote:
>>>>> On 6/10/2014 9:27 PM, Steve Campbell wrote:
>>>>>> On 6/10/2014 3:09 PM, Zoltan Frombach wrote:
>>>>>>> Steve,
>>>>>>>
>>>>>>> Try the following config.
>>>>>>>
>>>>>>> On your host:
>>>>>>>
>>>>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>>>>
>>>>>>> DEVICE=eth0
>>>>>>> BOOTPROTO=none
>>>>>>> ONBOOT=yes
>>>>>>> NM_CONTROLLED=no
>>>>>>> TYPE=Ethernet
>>>>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- put your physical NIC's MAC address
>>>>>>> here
>>>>>>> BRIDGE=br0
>>>>>>> USERCTL=no
>>>>>>> IPV6INIT=no
>>>>>>> IPV6_AUTOCONF=no
>>>>>>>
>>>>>>>
>>>>>>> /etc/sysconfig/network-scripts/ifcfg-br0    :
>>>>>>>
>>>>>>> DEVICE=br0
>>>>>>> BOOTPROTO=static
>>>>>>> ONBOOT=yes
>>>>>>> NM_CONTROLLED=no
>>>>>>> TYPE=Bridge
>>>>>>> IPADDR=10.0.5.16
>>>>>>> NETMASK=255.255.255.0   <- you may need to adjust this to your
>>>>>>> network
>>>>>>> GATEWAY=10.0.5.1
>>>>>>> DNS1=8.8.8.8
>>>>>>> DNS2=8.8.4.4
>>>>>>> PEERDNS=yes
>>>>>>> DELAY=0
>>>>>>> STP=off
>>>>>>> USERCTL=no
>>>>>>> IPV6INIT=no
>>>>>>> IPV6_AUTOCONF=no
>>>>>>>
>>>>>>>
>>>>>>> Restart networking on the host.
>>>>>>>
>>>>>>> Then inside you VM:
>>>>>>>
>>>>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>>>>
>>>>>>> DEVICE=eth0
>>>>>>> BOOTPROTO=static
>>>>>>> ONBOOT=yes
>>>>>>> NM_CONTROLLED=no
>>>>>>> TYPE=Ethernet
>>>>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- it must be a unique MAC address for
>>>>>>> your VM
>>>>>>> IPADDR=10.0.5.17
>>>>>>> NETMASK=255.255.255.0     <- you may need to adjust this to your
>>>>>>> network
>>>>>>> GATEWAY=10.0.5.1
>>>>>>> DNS1=8.8.8.8
>>>>>>> DNS2=8.8.4.4
>>>>>>> PEERDNS=yes
>>>>>>> USERCTL=no
>>>>>>> IPV6INIT=no
>>>>>>> IPV6_AUTOCONF=no
>>>>>>>
>>>>>>>
>>>>>>> Then in Virtual Machine Manager make sure that your VM's NIC is
>>>>>>> connected to the br0 bridge, like this:
>>>>>>>
>>>>>>> Network Source: Specify shared device name
>>>>>>>          Bridge Name: br0
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 6/10/2014 8:16 PM, Steve Campbell wrote:
>>>>>>>> On 6/10/2014 12:43 PM, Digimer wrote:
>>>>>>>>> On 10/06/14 12:38 PM, Steve Campbell wrote:
>>>>>>>>>> On 6/10/2014 12:05 PM, Digimer wrote:
>>>>>>>>>>> On 10/06/14 11:46 AM, Steve Campbell wrote:
>>>>>>>>>>>> On 6/10/2014 10:46 AM, Digimer wrote:
>>>>>>>>>>>>> On 10/06/14 10:03 AM, Steve Campbell wrote:
>>>>>>>>>>>>>> I had so much trouble putting Centos 6 guest VMs on a 
>>>>>>>>>>>>>> Centos 5
>>>>>>>>>>>>>> host
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> I finally switched to a Centos 6 host.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I've not needed more that test VMs, so I've used Virtual
>>>>>>>>>>>>>> Machine
>>>>>>>>>>>>>> Manager
>>>>>>>>>>>>>> on the old system, which worked pretty well, so I decided to
>>>>>>>>>>>>>> create my
>>>>>>>>>>>>>> first KVM guest machine. I noticed when I created it, I only
>>>>>>>>>>>>>> had the
>>>>>>>>>>>>>> options of NAT for my network interface, so I used that
>>>>>>>>>>>>>> (obvious).
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Well, after starting the VM, I find I don't have 
>>>>>>>>>>>>>> connectivity
>>>>>>>>>>>>>> with
>>>>>>>>>>>>>> that
>>>>>>>>>>>>>> interface. Reading, I find examples where I need to create
>>>>>>>>>>>>>> bridges
>>>>>>>>>>>>>> perhaps. Xen did most of this for me, so it's a little 
>>>>>>>>>>>>>> new to
>>>>>>>>>>>>>> me.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Can anyone throw me a clue, please?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> steve campbell
>>>>>>>>>>>>> Setting up a bridge is not that hard, and it will give 
>>>>>>>>>>>>> your VMs
>>>>>>>>>>>>> direct
>>>>>>>>>>>>> access to the outside world, and host <-> VM access just
>>>>>>>>>>>>> fine as
>>>>>>>>>>>>> well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Here is a link showing how to setup a bridge connected to a
>>>>>>>>>>>>> bond
>>>>>>>>>>>>> device. Ignore the bond and pretend it is a straight ethX
>>>>>>>>>>>>> device:
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://alteeve.ca/w/AN!Cluster_Tutorial_2#Configuring_our_Bridge.2C_Bonds_and_Interfaces 
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> The host has a device named virbr0 that is installed during
>>>>>>>>>>>> system
>>>>>>>>>>>> installation. It also has a network device vnet0. There are no
>>>>>>>>>>>> files in
>>>>>>>>>>>> /etc/sysconfig/network-scripts for these.
>>>>>>>>>>>>
>>>>>>>>>>>> Shouldn't I be able to use the virbr0 virtual bridge for this?
>>>>>>>>>>>> I've
>>>>>>>>>>>> tried setting up the VM's device with all of the options 
>>>>>>>>>>>> that is
>>>>>>>>>>>> listed,
>>>>>>>>>>>> but to no avail.
>>>>>>>>>>>>
>>>>>>>>>>>> Should I need to set up another bridge for this?
>>>>>>>>>>>>
>>>>>>>>>>>> And thanks for the link.
>>>>>>>>>>>>
>>>>>>>>>>>> steve
>>>>>>>>>>> virbr0 is created and managed by libvirtd. If you open Virtual
>>>>>>>>>>> Machine
>>>>>>>>>>> Manager, connect to localhost and then double-click on
>>>>>>>>>>> 'localhost',
>>>>>>>>>>> you will see a tab for creating/managing bridges (NAT'ed,
>>>>>>>>>>> generally).
>>>>>>>>>>> I disable 'virbr0' as NAT'ing is generally not what I want.
>>>>>>>>>>>
>>>>>>>>>>> The 'vnetX' devices are dynamically created to link a VM's
>>>>>>>>>>> interface
>>>>>>>>>>> to a bridge. Think of them as virtual network cables. They get
>>>>>>>>>>> created
>>>>>>>>>>> and destroyed as needed.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Sorry, but I'm confused:
>>>>>>>>>>
>>>>>>>>>> My host server has a real NIC and IP address with a real
>>>>>>>>>> gateway to
>>>>>>>>>> the
>>>>>>>>>> outside:
>>>>>>>>>> virtbr0 IP: 192.168.122.1
>>>>>>>>>> Host IP: 10.0.5.16
>>>>>>>>>> Gateway IP: 10.0.5.1
>>>>>>>>>> on eth0 and this works
>>>>>>>>>>
>>>>>>>>>> My VM server has all fake stuff currently:
>>>>>>>>>> Host IP: 10.0.5.17
>>>>>>>>>> Gateway IP: 10.0.5.1
>>>>>>>>>> on eth0 and this is like NIC without a cable.
>>>>>>>>>>
>>>>>>>>>> So I need to create a bridge device on both the host and VM 
>>>>>>>>>> (lets
>>>>>>>>>> say I
>>>>>>>>>> name it br1). I change the eth0 config file on both host and 
>>>>>>>>>> VM to
>>>>>>>>>> point
>>>>>>>>>> to br1 and give the br1 config file on both host and VM the
>>>>>>>>>> correct
>>>>>>>>>> IP.
>>>>>>>>>> But won't this just let the two talk to each other. How will
>>>>>>>>>> the VM
>>>>>>>>>> server get outside?
>>>>>>>>>>
>>>>>>>>>> steve
>>>>>>>>> The bridge is created only on the host. Think of the bridge as
>>>>>>>>> being
>>>>>>>>> like a virtual switch. When a VM is created, you tell it to
>>>>>>>>> connect to
>>>>>>>>> the bridge, similar to how you would plug a physical wire into a
>>>>>>>>> real
>>>>>>>>> switch. That provides the link to the network, and then you
>>>>>>>>> configure
>>>>>>>>> the virtual server's network just as if it was a real network.
>>>>>>>>>
>>>>>>>>> On the host, you don't set the IP on the ethX device, instead you
>>>>>>>>> tell
>>>>>>>>> ethX to connect to the bridge (look at 'bond2' in the link above
>>>>>>>>> for
>>>>>>>>> the 'BRIDGE="..."' line). Then you move the host's IP
>>>>>>>>> address/config
>>>>>>>>> to the bridge itself (look at the 'vbr2' in the link above).
>>>>>>>>>
>>>>>>>>> If you're still stuck, tell me the IP addresses you want to 
>>>>>>>>> set in
>>>>>>>>> the
>>>>>>>>> host and VM and I'll see if I can bang out a couple sample
>>>>>>>>> 'ifcfg-X'
>>>>>>>>> files.
>>>>>>>>>
>>>>>>>> I'm running into some problems with my config files where I get a
>>>>>>>> message that says the network can't determine my interface names.
>>>>>>>> I'll
>>>>>>>> look further, but the IPs above are what I need. 10.0.5.1 is a
>>>>>>>> firewall
>>>>>>>> that NATs the 10.0.5.0/24 network to another outside public
>>>>>>>> firewall.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> steve
>>>>>>>> _______________________________________________
>>>>>>>> CentOS-virt mailing list
>>>>>>>> CentOS-virt at centos.org
>>>>>>>> http://lists.centos.org/mailman/listinfo/centos-virt
>>>>>>>>
>>>>>> Stuck in a rut now. Restarting the network indicates :
>>>>>>
>>>>>> Device eth0 does not seem to be present, delaying initialization.
>>>>>>
>>>>>> I thought it might be due to the fact I didn't have the "TYPE" in 
>>>>>> the
>>>>>> ifcfg-eth0 file, but after adding that, I still get the error.
>>>>> NM_CONTROLLED=no might help with that.
>>>> I turned off and stopped NetworkManager and had already specified 
>>>> this.
>>>>>> I'm using the same hardware address that was in the original eth0
>>>>>> file.
>>>>> That is fine.
>>>>>
>>>>>>     My bridge device starts fine, but I'll tailor it to your specs.
>>>>> Also, you might want to check the contents of your
>>>>> /etc/udev/rules.d/70-persistent-net.rules file. Is the MAC address 
>>>>> for
>>>>> eth0 the same as in your /etc/sysconfig/network-scripts/ifcfg-eth0
>>>>> file?
>>>>>
>>>> They are the same except the udev file has lower case letters in it.
>>> You mean the hexadecimal numbers are lower case? That shouldn't matter.
>>>
>>> Try to delete the file /etc/udev/rules.d/70-persistent-net.rules and
>>> reboot your host machine.
>>> ( See
>>> http://www.andrewklau.com/device-eth0-does-not-seem-to-be-present-delaying-initialization/ 
>>>
>>> )
>>>
>> All is well on the host now, and I've brought up the VM and configured
>> it to use the bridge. I can ping the host from the VM but I have no
>> access beyond the local network. The host sees the world, but the VM
>> only sees the 10.0.5.0/24 network.
> Did you set the gateway in your VM to 10.0.5.1 ?
> Can you ping the gateway ( ping 10.0.5.1 ) from the VM?
> Do you have a firewall configured in your VM (or host) ? If so, try to 
> disable the firewall temporarily for testing.
>
The gateway is set. Can't ping the gateway. Firewalls (iptables and 
ip6tables) are stopped and off. Previous post suggested the route. I'll 
check it.

steve