[CentOS-virt] Finally switching from Xen to KVM - question about networking

Tue Jun 10 20:00:45 UTC 2014
Zoltan Frombach <zoltan at frombach.com>

On 6/10/2014 9:51 PM, Steve Campbell wrote:
> On 6/10/2014 3:38 PM, Zoltan Frombach wrote:
>> On 6/10/2014 9:27 PM, Steve Campbell wrote:
>>> On 6/10/2014 3:09 PM, Zoltan Frombach wrote:
>>>> Steve,
>>>>
>>>> Try the following config.
>>>>
>>>> On your host:
>>>>
>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>
>>>> DEVICE=eth0
>>>> BOOTPROTO=none
>>>> ONBOOT=yes
>>>> NM_CONTROLLED=no
>>>> TYPE=Ethernet
>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- put your physical NIC's MAC address here
>>>> BRIDGE=br0
>>>> USERCTL=no
>>>> IPV6INIT=no
>>>> IPV6_AUTOCONF=no
>>>>
>>>>
>>>> /etc/sysconfig/network-scripts/ifcfg-br0    :
>>>>
>>>> DEVICE=br0
>>>> BOOTPROTO=static
>>>> ONBOOT=yes
>>>> NM_CONTROLLED=no
>>>> TYPE=Bridge
>>>> IPADDR=10.0.5.16
>>>> NETMASK=255.255.255.0   <- you may need to adjust this to your network
>>>> GATEWAY=10.0.5.1
>>>> DNS1=8.8.8.8
>>>> DNS2=8.8.4.4
>>>> PEERDNS=yes
>>>> DELAY=0
>>>> STP=off
>>>> USERCTL=no
>>>> IPV6INIT=no
>>>> IPV6_AUTOCONF=no
>>>>
>>>>
>>>> Restart networking on the host.
>>>>
>>>> Then inside you VM:
>>>>
>>>> /etc/sysconfig/network-scripts/ifcfg-eth0    :
>>>>
>>>> DEVICE=eth0
>>>> BOOTPROTO=static
>>>> ONBOOT=yes
>>>> NM_CONTROLLED=no
>>>> TYPE=Ethernet
>>>> HWADDR=xx:xx:xx:xx:xx:xx    <- it must be a unique MAC address for
>>>> your VM
>>>> IPADDR=10.0.5.17
>>>> NETMASK=255.255.255.0     <- you may need to adjust this to your
>>>> network
>>>> GATEWAY=10.0.5.1
>>>> DNS1=8.8.8.8
>>>> DNS2=8.8.4.4
>>>> PEERDNS=yes
>>>> USERCTL=no
>>>> IPV6INIT=no
>>>> IPV6_AUTOCONF=no
>>>>
>>>>
>>>> Then in Virtual Machine Manager make sure that your VM's NIC is
>>>> connected to the br0 bridge, like this:
>>>>
>>>> Network Source: Specify shared device name
>>>>         Bridge Name: br0
>>>>
>>>>
>>>>
>>>> On 6/10/2014 8:16 PM, Steve Campbell wrote:
>>>>> On 6/10/2014 12:43 PM, Digimer wrote:
>>>>>> On 10/06/14 12:38 PM, Steve Campbell wrote:
>>>>>>> On 6/10/2014 12:05 PM, Digimer wrote:
>>>>>>>> On 10/06/14 11:46 AM, Steve Campbell wrote:
>>>>>>>>> On 6/10/2014 10:46 AM, Digimer wrote:
>>>>>>>>>> On 10/06/14 10:03 AM, Steve Campbell wrote:
>>>>>>>>>>> I had so much trouble putting Centos 6 guest VMs on a Centos 5
>>>>>>>>>>> host
>>>>>>>>>>> that
>>>>>>>>>>> I finally switched to a Centos 6 host.
>>>>>>>>>>>
>>>>>>>>>>> I've not needed more that test VMs, so I've used Virtual Machine
>>>>>>>>>>> Manager
>>>>>>>>>>> on the old system, which worked pretty well, so I decided to
>>>>>>>>>>> create my
>>>>>>>>>>> first KVM guest machine. I noticed when I created it, I only
>>>>>>>>>>> had the
>>>>>>>>>>> options of NAT for my network interface, so I used that
>>>>>>>>>>> (obvious).
>>>>>>>>>>>
>>>>>>>>>>> Well, after starting the VM, I find I don't have connectivity
>>>>>>>>>>> with
>>>>>>>>>>> that
>>>>>>>>>>> interface. Reading, I find examples where I need to create
>>>>>>>>>>> bridges
>>>>>>>>>>> perhaps. Xen did most of this for me, so it's a little new to
>>>>>>>>>>> me.
>>>>>>>>>>>
>>>>>>>>>>> Can anyone throw me a clue, please?
>>>>>>>>>>>
>>>>>>>>>>> steve campbell
>>>>>>>>>> Setting up a bridge is not that hard, and it will give your VMs
>>>>>>>>>> direct
>>>>>>>>>> access to the outside world, and host <-> VM access just fine as
>>>>>>>>>> well.
>>>>>>>>>>
>>>>>>>>>> Here is a link showing how to setup a bridge connected to a bond
>>>>>>>>>> device. Ignore the bond and pretend it is a straight ethX device:
>>>>>>>>>>
>>>>>>>>>> https://alteeve.ca/w/AN!Cluster_Tutorial_2#Configuring_our_Bridge.2C_Bonds_and_Interfaces
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> The host has a device named virbr0 that is installed during system
>>>>>>>>> installation. It also has a network device vnet0. There are no
>>>>>>>>> files in
>>>>>>>>> /etc/sysconfig/network-scripts for these.
>>>>>>>>>
>>>>>>>>> Shouldn't I be able to use the virbr0 virtual bridge for this?
>>>>>>>>> I've
>>>>>>>>> tried setting up the VM's device with all of the options that is
>>>>>>>>> listed,
>>>>>>>>> but to no avail.
>>>>>>>>>
>>>>>>>>> Should I need to set up another bridge for this?
>>>>>>>>>
>>>>>>>>> And thanks for the link.
>>>>>>>>>
>>>>>>>>> steve
>>>>>>>> virbr0 is created and managed by libvirtd. If you open Virtual
>>>>>>>> Machine
>>>>>>>> Manager, connect to localhost and then double-click on 'localhost',
>>>>>>>> you will see a tab for creating/managing bridges (NAT'ed,
>>>>>>>> generally).
>>>>>>>> I disable 'virbr0' as NAT'ing is generally not what I want.
>>>>>>>>
>>>>>>>> The 'vnetX' devices are dynamically created to link a VM's
>>>>>>>> interface
>>>>>>>> to a bridge. Think of them as virtual network cables. They get
>>>>>>>> created
>>>>>>>> and destroyed as needed.
>>>>>>>>
>>>>>>>>
>>>>>>> Sorry, but I'm confused:
>>>>>>>
>>>>>>> My host server has a real NIC and IP address with a real gateway to
>>>>>>> the
>>>>>>> outside:
>>>>>>> virtbr0 IP: 192.168.122.1
>>>>>>> Host IP: 10.0.5.16
>>>>>>> Gateway IP: 10.0.5.1
>>>>>>> on eth0 and this works
>>>>>>>
>>>>>>> My VM server has all fake stuff currently:
>>>>>>> Host IP: 10.0.5.17
>>>>>>> Gateway IP: 10.0.5.1
>>>>>>> on eth0 and this is like NIC without a cable.
>>>>>>>
>>>>>>> So I need to create a bridge device on both the host and VM (lets
>>>>>>> say I
>>>>>>> name it br1). I change the eth0 config file on both host and VM to
>>>>>>> point
>>>>>>> to br1 and give the br1 config file on both host and VM the correct
>>>>>>> IP.
>>>>>>> But won't this just let the two talk to each other. How will the VM
>>>>>>> server get outside?
>>>>>>>
>>>>>>> steve
>>>>>> The bridge is created only on the host. Think of the bridge as being
>>>>>> like a virtual switch. When a VM is created, you tell it to
>>>>>> connect to
>>>>>> the bridge, similar to how you would plug a physical wire into a real
>>>>>> switch. That provides the link to the network, and then you configure
>>>>>> the virtual server's network just as if it was a real network.
>>>>>>
>>>>>> On the host, you don't set the IP on the ethX device, instead you
>>>>>> tell
>>>>>> ethX to connect to the bridge (look at 'bond2' in the link above for
>>>>>> the 'BRIDGE="..."' line). Then you move the host's IP address/config
>>>>>> to the bridge itself (look at the 'vbr2' in the link above).
>>>>>>
>>>>>> If you're still stuck, tell me the IP addresses you want to set in
>>>>>> the
>>>>>> host and VM and I'll see if I can bang out a couple sample 'ifcfg-X'
>>>>>> files.
>>>>>>
>>>>> I'm running into some problems with my config files where I get a
>>>>> message that says the network can't determine my interface names. I'll
>>>>> look further, but the IPs above are what I need. 10.0.5.1 is a
>>>>> firewall
>>>>> that NATs the 10.0.5.0/24 network to another outside public firewall.
>>>>>
>>>>> Thanks
>>>>> steve
>>>>> _______________________________________________
>>>>> CentOS-virt mailing list
>>>>> CentOS-virt at centos.org
>>>>> http://lists.centos.org/mailman/listinfo/centos-virt
>>>>>
>>> Stuck in a rut now. Restarting the network indicates :
>>>
>>> Device eth0 does not seem to be present, delaying initialization.
>>>
>>> I thought it might be due to the fact I didn't have the "TYPE" in the
>>> ifcfg-eth0 file, but after adding that, I still get the error.
>> NM_CONTROLLED=no might help with that.
> I turned off and stopped NetworkManager and had already specified this.
>>> I'm using the same hardware address that was in the original eth0 file.
>> That is fine.
>>
>>>    My bridge device starts fine, but I'll tailor it to your specs.
>> Also, you might want to check the contents of your
>> /etc/udev/rules.d/70-persistent-net.rules file. Is the MAC address for
>> eth0 the same as in your /etc/sysconfig/network-scripts/ifcfg-eth0 file?
>>
> They are the same except the udev file has lower case letters in it.

You mean the hexadecimal numbers are lower case? That shouldn't matter.

Try to delete the file /etc/udev/rules.d/70-persistent-net.rules and 
reboot your host machine.
( See 
http://www.andrewklau.com/device-eth0-does-not-seem-to-be-present-delaying-initialization/ 
)