[CentOS-virt] Finally switching from Xen to KVM - question about networking

Tue Jun 10 19:51:06 UTC 2014
Steve Campbell <campbell at cnpapers.com>

On 6/10/2014 3:38 PM, Zoltan Frombach wrote:
>
> On 6/10/2014 9:27 PM, Steve Campbell wrote:
>> On 6/10/2014 3:09 PM, Zoltan Frombach wrote:
>>> Steve,
>>>
>>> Try the following config.
>>>
>>> On your host:
>>>
>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>
>>> DEVICE=eth0
>>> BOOTPROTO=none
>>> ONBOOT=yes
>>> NM_CONTROLLED=no
>>> TYPE=Ethernet
>>> HWADDR=xx:xx:xx:xx:xx:xx    <- put your physical NIC's MAC address here
>>> BRIDGE=br0
>>> USERCTL=no
>>> IPV6INIT=no
>>> IPV6_AUTOCONF=no
>>>
>>>
>>> /etc/sysconfig/network-scripts/ifcfg-br0    :
>>>
>>> DEVICE=br0
>>> BOOTPROTO=static
>>> ONBOOT=yes
>>> NM_CONTROLLED=no
>>> TYPE=Bridge
>>> IPADDR=10.0.5.16
>>> NETMASK=255.255.255.0   <- you may need to adjust this to your network
>>> GATEWAY=10.0.5.1
>>> DNS1=8.8.8.8
>>> DNS2=8.8.4.4
>>> PEERDNS=yes
>>> DELAY=0
>>> STP=off
>>> USERCTL=no
>>> IPV6INIT=no
>>> IPV6_AUTOCONF=no
>>>
>>>
>>> Restart networking on the host.
>>>
>>> Then inside you VM:
>>>
>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>
>>> DEVICE=eth0
>>> BOOTPROTO=static
>>> ONBOOT=yes
>>> NM_CONTROLLED=no
>>> TYPE=Ethernet
>>> HWADDR=xx:xx:xx:xx:xx:xx    <- it must be a unique MAC address for
>>> your VM
>>> IPADDR=10.0.5.17
>>> NETMASK=255.255.255.0     <- you may need to adjust this to your 
>>> network
>>> GATEWAY=10.0.5.1
>>> DNS1=8.8.8.8
>>> DNS2=8.8.4.4
>>> PEERDNS=yes
>>> USERCTL=no
>>> IPV6INIT=no
>>> IPV6_AUTOCONF=no
>>>
>>>
>>> Then in Virtual Machine Manager make sure that your VM's NIC is
>>> connected to the br0 bridge, like this:
>>>
>>> Network Source: Specify shared device name
>>>        Bridge Name: br0
>>>
>>>
>>>
>>> On 6/10/2014 8:16 PM, Steve Campbell wrote:
>>>> On 6/10/2014 12:43 PM, Digimer wrote:
>>>>> On 10/06/14 12:38 PM, Steve Campbell wrote:
>>>>>> On 6/10/2014 12:05 PM, Digimer wrote:
>>>>>>> On 10/06/14 11:46 AM, Steve Campbell wrote:
>>>>>>>> On 6/10/2014 10:46 AM, Digimer wrote:
>>>>>>>>> On 10/06/14 10:03 AM, Steve Campbell wrote:
>>>>>>>>>> I had so much trouble putting Centos 6 guest VMs on a Centos 5
>>>>>>>>>> host
>>>>>>>>>> that
>>>>>>>>>> I finally switched to a Centos 6 host.
>>>>>>>>>>
>>>>>>>>>> I've not needed more that test VMs, so I've used Virtual Machine
>>>>>>>>>> Manager
>>>>>>>>>> on the old system, which worked pretty well, so I decided to
>>>>>>>>>> create my
>>>>>>>>>> first KVM guest machine. I noticed when I created it, I only
>>>>>>>>>> had the
>>>>>>>>>> options of NAT for my network interface, so I used that 
>>>>>>>>>> (obvious).
>>>>>>>>>>
>>>>>>>>>> Well, after starting the VM, I find I don't have connectivity 
>>>>>>>>>> with
>>>>>>>>>> that
>>>>>>>>>> interface. Reading, I find examples where I need to create 
>>>>>>>>>> bridges
>>>>>>>>>> perhaps. Xen did most of this for me, so it's a little new to 
>>>>>>>>>> me.
>>>>>>>>>>
>>>>>>>>>> Can anyone throw me a clue, please?
>>>>>>>>>>
>>>>>>>>>> steve campbell
>>>>>>>>> Setting up a bridge is not that hard, and it will give your VMs
>>>>>>>>> direct
>>>>>>>>> access to the outside world, and host <-> VM access just fine as
>>>>>>>>> well.
>>>>>>>>>
>>>>>>>>> Here is a link showing how to setup a bridge connected to a bond
>>>>>>>>> device. Ignore the bond and pretend it is a straight ethX device:
>>>>>>>>>
>>>>>>>>> https://alteeve.ca/w/AN!Cluster_Tutorial_2#Configuring_our_Bridge.2C_Bonds_and_Interfaces 
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>> The host has a device named virbr0 that is installed during system
>>>>>>>> installation. It also has a network device vnet0. There are no
>>>>>>>> files in
>>>>>>>> /etc/sysconfig/network-scripts for these.
>>>>>>>>
>>>>>>>> Shouldn't I be able to use the virbr0 virtual bridge for this? 
>>>>>>>> I've
>>>>>>>> tried setting up the VM's device with all of the options that is
>>>>>>>> listed,
>>>>>>>> but to no avail.
>>>>>>>>
>>>>>>>> Should I need to set up another bridge for this?
>>>>>>>>
>>>>>>>> And thanks for the link.
>>>>>>>>
>>>>>>>> steve
>>>>>>> virbr0 is created and managed by libvirtd. If you open Virtual
>>>>>>> Machine
>>>>>>> Manager, connect to localhost and then double-click on 'localhost',
>>>>>>> you will see a tab for creating/managing bridges (NAT'ed, 
>>>>>>> generally).
>>>>>>> I disable 'virbr0' as NAT'ing is generally not what I want.
>>>>>>>
>>>>>>> The 'vnetX' devices are dynamically created to link a VM's 
>>>>>>> interface
>>>>>>> to a bridge. Think of them as virtual network cables. They get
>>>>>>> created
>>>>>>> and destroyed as needed.
>>>>>>>
>>>>>>>
>>>>>> Sorry, but I'm confused:
>>>>>>
>>>>>> My host server has a real NIC and IP address with a real gateway to
>>>>>> the
>>>>>> outside:
>>>>>> virtbr0 IP: 192.168.122.1
>>>>>> Host IP: 10.0.5.16
>>>>>> Gateway IP: 10.0.5.1
>>>>>> on eth0 and this works
>>>>>>
>>>>>> My VM server has all fake stuff currently:
>>>>>> Host IP: 10.0.5.17
>>>>>> Gateway IP: 10.0.5.1
>>>>>> on eth0 and this is like NIC without a cable.
>>>>>>
>>>>>> So I need to create a bridge device on both the host and VM (lets
>>>>>> say I
>>>>>> name it br1). I change the eth0 config file on both host and VM to
>>>>>> point
>>>>>> to br1 and give the br1 config file on both host and VM the correct
>>>>>> IP.
>>>>>> But won't this just let the two talk to each other. How will the VM
>>>>>> server get outside?
>>>>>>
>>>>>> steve
>>>>> The bridge is created only on the host. Think of the bridge as being
>>>>> like a virtual switch. When a VM is created, you tell it to 
>>>>> connect to
>>>>> the bridge, similar to how you would plug a physical wire into a real
>>>>> switch. That provides the link to the network, and then you configure
>>>>> the virtual server's network just as if it was a real network.
>>>>>
>>>>> On the host, you don't set the IP on the ethX device, instead you 
>>>>> tell
>>>>> ethX to connect to the bridge (look at 'bond2' in the link above for
>>>>> the 'BRIDGE="..."' line). Then you move the host's IP address/config
>>>>> to the bridge itself (look at the 'vbr2' in the link above).
>>>>>
>>>>> If you're still stuck, tell me the IP addresses you want to set in 
>>>>> the
>>>>> host and VM and I'll see if I can bang out a couple sample 'ifcfg-X'
>>>>> files.
>>>>>
>>>> I'm running into some problems with my config files where I get a
>>>> message that says the network can't determine my interface names. I'll
>>>> look further, but the IPs above are what I need. 10.0.5.1 is a 
>>>> firewall
>>>> that NATs the 10.0.5.0/24 network to another outside public firewall.
>>>>
>>>> Thanks
>>>> steve
>>>> _______________________________________________
>>>> CentOS-virt mailing list
>>>> CentOS-virt at centos.org
>>>> http://lists.centos.org/mailman/listinfo/centos-virt
>>>>
>> Stuck in a rut now. Restarting the network indicates :
>>
>> Device eth0 does not seem to be present, delaying initialization.
>>
>> I thought it might be due to the fact I didn't have the "TYPE" in the
>> ifcfg-eth0 file, but after adding that, I still get the error.
>
> NM_CONTROLLED=no might help with that.

I turned off and stopped NetworkManager and had already specified this.
>
>> I'm using the same hardware address that was in the original eth0 file.
>
> That is fine.
>
>>   My bridge device starts fine, but I'll tailor it to your specs.
>
> Also, you might want to check the contents of your 
> /etc/udev/rules.d/70-persistent-net.rules file. Is the MAC address for 
> eth0 the same as in your /etc/sysconfig/network-scripts/ifcfg-eth0 file?
>
They are the same except the udev file has lower case letters in it.
I appreciate your time on this.

steve