[CentOS-virt] Finally switching from Xen to KVM - question about networking

Tue Jun 10 20:09:28 UTC 2014
Steve Campbell <campbell at cnpapers.com>

On 6/10/2014 4:00 PM, Zoltan Frombach wrote:
>
> On 6/10/2014 9:51 PM, Steve Campbell wrote:
>> On 6/10/2014 3:38 PM, Zoltan Frombach wrote:
>>> On 6/10/2014 9:27 PM, Steve Campbell wrote:
>>>> On 6/10/2014 3:09 PM, Zoltan Frombach wrote:
>>>>> Steve,
>>>>>
>>>>> Try the following config.
>>>>>
>>>>> On your host:
>>>>>
>>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>>
>>>>> DEVICE=eth0
>>>>> BOOTPROTO=none
>>>>> ONBOOT=yes
>>>>> NM_CONTROLLED=no
>>>>> TYPE=Ethernet
>>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- put your physical NIC's MAC address 
>>>>> here
>>>>> BRIDGE=br0
>>>>> USERCTL=no
>>>>> IPV6INIT=no
>>>>> IPV6_AUTOCONF=no
>>>>>
>>>>>
>>>>> /etc/sysconfig/network-scripts/ifcfg-br0    :
>>>>>
>>>>> DEVICE=br0
>>>>> BOOTPROTO=static
>>>>> ONBOOT=yes
>>>>> NM_CONTROLLED=no
>>>>> TYPE=Bridge
>>>>> IPADDR=10.0.5.16
>>>>> NETMASK=255.255.255.0   <- you may need to adjust this to your 
>>>>> network
>>>>> GATEWAY=10.0.5.1
>>>>> DNS1=8.8.8.8
>>>>> DNS2=8.8.4.4
>>>>> PEERDNS=yes
>>>>> DELAY=0
>>>>> STP=off
>>>>> USERCTL=no
>>>>> IPV6INIT=no
>>>>> IPV6_AUTOCONF=no
>>>>>
>>>>>
>>>>> Restart networking on the host.
>>>>>
>>>>> Then inside you VM:
>>>>>
>>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>>
>>>>> DEVICE=eth0
>>>>> BOOTPROTO=static
>>>>> ONBOOT=yes
>>>>> NM_CONTROLLED=no
>>>>> TYPE=Ethernet
>>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- it must be a unique MAC address for
>>>>> your VM
>>>>> IPADDR=10.0.5.17
>>>>> NETMASK=255.255.255.0     <- you may need to adjust this to your
>>>>> network
>>>>> GATEWAY=10.0.5.1
>>>>> DNS1=8.8.8.8
>>>>> DNS2=8.8.4.4
>>>>> PEERDNS=yes
>>>>> USERCTL=no
>>>>> IPV6INIT=no
>>>>> IPV6_AUTOCONF=no
>>>>>
>>>>>
>>>>> Then in Virtual Machine Manager make sure that your VM's NIC is
>>>>> connected to the br0 bridge, like this:
>>>>>
>>>>> Network Source: Specify shared device name
>>>>>         Bridge Name: br0
>>>>>
>>>>>
>>>>>
>>>>> On 6/10/2014 8:16 PM, Steve Campbell wrote:
>>>>>> On 6/10/2014 12:43 PM, Digimer wrote:
>>>>>>> On 10/06/14 12:38 PM, Steve Campbell wrote:
>>>>>>>> On 6/10/2014 12:05 PM, Digimer wrote:
>>>>>>>>> On 10/06/14 11:46 AM, Steve Campbell wrote:
>>>>>>>>>> On 6/10/2014 10:46 AM, Digimer wrote:
>>>>>>>>>>> On 10/06/14 10:03 AM, Steve Campbell wrote:
>>>>>>>>>>>> I had so much trouble putting Centos 6 guest VMs on a Centos 5
>>>>>>>>>>>> host
>>>>>>>>>>>> that
>>>>>>>>>>>> I finally switched to a Centos 6 host.
>>>>>>>>>>>>
>>>>>>>>>>>> I've not needed more that test VMs, so I've used Virtual 
>>>>>>>>>>>> Machine
>>>>>>>>>>>> Manager
>>>>>>>>>>>> on the old system, which worked pretty well, so I decided to
>>>>>>>>>>>> create my
>>>>>>>>>>>> first KVM guest machine. I noticed when I created it, I only
>>>>>>>>>>>> had the
>>>>>>>>>>>> options of NAT for my network interface, so I used that
>>>>>>>>>>>> (obvious).
>>>>>>>>>>>>
>>>>>>>>>>>> Well, after starting the VM, I find I don't have connectivity
>>>>>>>>>>>> with
>>>>>>>>>>>> that
>>>>>>>>>>>> interface. Reading, I find examples where I need to create
>>>>>>>>>>>> bridges
>>>>>>>>>>>> perhaps. Xen did most of this for me, so it's a little new to
>>>>>>>>>>>> me.
>>>>>>>>>>>>
>>>>>>>>>>>> Can anyone throw me a clue, please?
>>>>>>>>>>>>
>>>>>>>>>>>> steve campbell
>>>>>>>>>>> Setting up a bridge is not that hard, and it will give your VMs
>>>>>>>>>>> direct
>>>>>>>>>>> access to the outside world, and host <-> VM access just 
>>>>>>>>>>> fine as
>>>>>>>>>>> well.
>>>>>>>>>>>
>>>>>>>>>>> Here is a link showing how to setup a bridge connected to a 
>>>>>>>>>>> bond
>>>>>>>>>>> device. Ignore the bond and pretend it is a straight ethX 
>>>>>>>>>>> device:
>>>>>>>>>>>
>>>>>>>>>>> https://alteeve.ca/w/AN!Cluster_Tutorial_2#Configuring_our_Bridge.2C_Bonds_and_Interfaces 
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> The host has a device named virbr0 that is installed during 
>>>>>>>>>> system
>>>>>>>>>> installation. It also has a network device vnet0. There are no
>>>>>>>>>> files in
>>>>>>>>>> /etc/sysconfig/network-scripts for these.
>>>>>>>>>>
>>>>>>>>>> Shouldn't I be able to use the virbr0 virtual bridge for this?
>>>>>>>>>> I've
>>>>>>>>>> tried setting up the VM's device with all of the options that is
>>>>>>>>>> listed,
>>>>>>>>>> but to no avail.
>>>>>>>>>>
>>>>>>>>>> Should I need to set up another bridge for this?
>>>>>>>>>>
>>>>>>>>>> And thanks for the link.
>>>>>>>>>>
>>>>>>>>>> steve
>>>>>>>>> virbr0 is created and managed by libvirtd. If you open Virtual
>>>>>>>>> Machine
>>>>>>>>> Manager, connect to localhost and then double-click on 
>>>>>>>>> 'localhost',
>>>>>>>>> you will see a tab for creating/managing bridges (NAT'ed,
>>>>>>>>> generally).
>>>>>>>>> I disable 'virbr0' as NAT'ing is generally not what I want.
>>>>>>>>>
>>>>>>>>> The 'vnetX' devices are dynamically created to link a VM's
>>>>>>>>> interface
>>>>>>>>> to a bridge. Think of them as virtual network cables. They get
>>>>>>>>> created
>>>>>>>>> and destroyed as needed.
>>>>>>>>>
>>>>>>>>>
>>>>>>>> Sorry, but I'm confused:
>>>>>>>>
>>>>>>>> My host server has a real NIC and IP address with a real 
>>>>>>>> gateway to
>>>>>>>> the
>>>>>>>> outside:
>>>>>>>> virtbr0 IP: 192.168.122.1
>>>>>>>> Host IP: 10.0.5.16
>>>>>>>> Gateway IP: 10.0.5.1
>>>>>>>> on eth0 and this works
>>>>>>>>
>>>>>>>> My VM server has all fake stuff currently:
>>>>>>>> Host IP: 10.0.5.17
>>>>>>>> Gateway IP: 10.0.5.1
>>>>>>>> on eth0 and this is like NIC without a cable.
>>>>>>>>
>>>>>>>> So I need to create a bridge device on both the host and VM (lets
>>>>>>>> say I
>>>>>>>> name it br1). I change the eth0 config file on both host and VM to
>>>>>>>> point
>>>>>>>> to br1 and give the br1 config file on both host and VM the 
>>>>>>>> correct
>>>>>>>> IP.
>>>>>>>> But won't this just let the two talk to each other. How will 
>>>>>>>> the VM
>>>>>>>> server get outside?
>>>>>>>>
>>>>>>>> steve
>>>>>>> The bridge is created only on the host. Think of the bridge as 
>>>>>>> being
>>>>>>> like a virtual switch. When a VM is created, you tell it to
>>>>>>> connect to
>>>>>>> the bridge, similar to how you would plug a physical wire into a 
>>>>>>> real
>>>>>>> switch. That provides the link to the network, and then you 
>>>>>>> configure
>>>>>>> the virtual server's network just as if it was a real network.
>>>>>>>
>>>>>>> On the host, you don't set the IP on the ethX device, instead you
>>>>>>> tell
>>>>>>> ethX to connect to the bridge (look at 'bond2' in the link above 
>>>>>>> for
>>>>>>> the 'BRIDGE="..."' line). Then you move the host's IP 
>>>>>>> address/config
>>>>>>> to the bridge itself (look at the 'vbr2' in the link above).
>>>>>>>
>>>>>>> If you're still stuck, tell me the IP addresses you want to set in
>>>>>>> the
>>>>>>> host and VM and I'll see if I can bang out a couple sample 
>>>>>>> 'ifcfg-X'
>>>>>>> files.
>>>>>>>
>>>>>> I'm running into some problems with my config files where I get a
>>>>>> message that says the network can't determine my interface names. 
>>>>>> I'll
>>>>>> look further, but the IPs above are what I need. 10.0.5.1 is a
>>>>>> firewall
>>>>>> that NATs the 10.0.5.0/24 network to another outside public 
>>>>>> firewall.
>>>>>>
>>>>>> Thanks
>>>>>> steve
>>>>>> _______________________________________________
>>>>>> CentOS-virt mailing list
>>>>>> CentOS-virt at centos.org
>>>>>> http://lists.centos.org/mailman/listinfo/centos-virt
>>>>>>
>>>> Stuck in a rut now. Restarting the network indicates :
>>>>
>>>> Device eth0 does not seem to be present, delaying initialization.
>>>>
>>>> I thought it might be due to the fact I didn't have the "TYPE" in the
>>>> ifcfg-eth0 file, but after adding that, I still get the error.
>>> NM_CONTROLLED=no might help with that.
>> I turned off and stopped NetworkManager and had already specified this.
>>>> I'm using the same hardware address that was in the original eth0 
>>>> file.
>>> That is fine.
>>>
>>>>    My bridge device starts fine, but I'll tailor it to your specs.
>>> Also, you might want to check the contents of your
>>> /etc/udev/rules.d/70-persistent-net.rules file. Is the MAC address for
>>> eth0 the same as in your /etc/sysconfig/network-scripts/ifcfg-eth0 
>>> file?
>>>
>> They are the same except the udev file has lower case letters in it.
>
> You mean the hexadecimal numbers are lower case? That shouldn't matter.
>
> Try to delete the file /etc/udev/rules.d/70-persistent-net.rules and 
> reboot your host machine.
> ( See 
> http://www.andrewklau.com/device-eth0-does-not-seem-to-be-present-delaying-initialization/ 
> )
>
Looks like that did it. Now on to the VM.

Thanks so much.

steve