[CentOS-virt] xenbr0 isn't created anymore

Fri Mar 28 16:20:54 UTC 2008
Ross S. W. Walker <rwalker at medallion.com>

Kai Schaetzl wrote:
> Ross S. W. Walker wrote on Fri, 28 Mar 2008 09:44:44 -0400:
> > 
> > Snooping around I found this in /etc/xen/qemu-ifup:
> 
> Interesting. That is the one from Xen 3.2 I suppose? It's not what I have 
> here with the mint CentOS 5.1 Xen.
> 
> I just have:
> 
> echo 'config qemu network with xen bridge for ' $*
> 
> 
> ifconfig $1 0.0.0.0 up
> 
> brctl addif $2 $1
> 
> That might be used like "qemu-ifup xenbr0 eth0" or so I guess. I don't know 
> if it would work but it sure never gets run (or if it runs it doesn't 
> work). There's no "config qemu network with xen bridge" in any of my logs, 
> so I'd say it never runs. And it should never run as I'm not using qemu. It 
> might indeed only be there for qemu to use.

Hmmm, how about in the /xen/scripts/vif-bridge script? Does it provide
those comments there?

It may be that the implementation of Xen in CentOS depends on the libvirtd
service installed and running and all VMs running off of virbr0?

> > #
> > # Old style bridge setup with netloop, used to have a bridge name
> > # of xenbrX, enslaving pethX and vif0.X, and then configuring
> > # eth0.
> > #
> > # New style bridge setup does not use netloop, so the bridge name
> > # is ethX and the physical device is enslaved pethX
> 
> Hm, but that sounds like what they do currently. peth0 is supposed to be 
> the physical device. But the bridge name is not eth0. So, it might be 
> indeed a new way. While doing research I saw a number of complaints about 
> their current way of doing it and saw much easier ways. So maybe they 
> changed it in this manner.

Here is the output of 'brctl show' on my box:

bridge name     bridge id               STP enabled     interfaces
eth0            8000.00188b717d72       no              vif2.0
                                                        tap0
                                                        peth0

On the older Xen it would look like this:

bridge name     bridge id               STP enabled     interfaces
xenbr0          8000.00188b717d72       no              vif0.0
                                                        vif2.0
                                                        tap0
                                                        peth0

Then vif0.0 would be netlooped to eth0, so it is much simplier
now.

> > #
> > # So if...
> > #
> > #   - User asks for xenbrX
> > #   - AND xenbrX doesn't exist
> > #   - AND there is a ethX device which is a bridge
> > #
> > # ..then we translate xenbrX to ethX
> 
> That would probably cut it.
> 
> > Now I am not sure how the vif0.X and vethX interfaces fit into the
> > picture here... Maybe these are no longer used?
> 
> Still there I would think, just that xenbr0 (after creation) bridges to 
> eth0 and not peth0. I suppose. In ifconfig it will just look the same as 
> before. I suppose. Again.

Actually you misread, xenbr0 was removed, now eth0 is the bridge itself.

> I found a cure for the problem yesterday evening. That bridge-network 
> script is crap. It depends on certain output from "ip route list". What 
> format do you get as the last line of a "ip route list"?
> 
> default via 192.168.1.1 dev eth0  src 192.168.1.231
> 
> or
> 
> default via 192.168.1.1 dev eth0

The second one:

default via 10.1.3.1 dev eth0

> I get this appended src on any of my 5.1 setups but not on my 4.6 setups. I 
> don't have a 5.0 for comparison handy. The date of the ip file is from 
> March 2007, so I'd say it didn't get updated recently. But maybe some 
> library that is involved in this got updated. As it used to work until some 
> days ago either some update stopped it working or I somehow stopped the 
> qemu-ifup (in case it ever really fixed this) from running or from running 
> correctly. Or yet something else ...

I believe you get that src 192.168.1.231 address if that route is discovered
via ICMP router discovery. If it was given to you in DHCP or if you specified
it in your ifcfg script with GATEWAY= then that probably wouldn't appear.

> This is so hard to troubleshoot because those scripts won't add any logging 
> to the normal syslog. If at all logging goes to xend.log or xend-debug.log 
> and doesn't have any date/time attached, so it's easily mistaken as an 
> error that occurred from running the script manually (which I did often 
> enough).

Not only that, but they appear to be Xen 3.0.3 scripts... which are quite
old.

> Here's the beginning of the code in network-bridge:
> 
> #vifnum=${vifnum:-$(ip route list | awk '/^default / { print $NF }' | sed 's/^[^0-9]*//')}
> vifnum=${vifnum:-0}
> #bridge=${bridge:-xenbr${vifnum}}
> bridge=xenbr0
> #netdev=${netdev:-eth${vifnum}}
> netdev=eth0
> antispoof=${antispoof:-no}
> 
> I added the comments and the two lines that set bridge and netdev to fixed 
> values. Now it works. Without this it would get "192.168.1.231" as $vifnum 
> and try to use devices like eth192.168.1.231 and xenbr192.168.1.231. With 
> the old output of "ip route list" it would grab "0" instead of 
> "192.168.1.231" and thus use the correct interface names.
> With this change I can now start and stop xend and it will take up and down 
> the xenbr0 and connected devices correctly. Before it would just silently 
> fail. (Or work for starting only once I killed eth0 as then it would take 
> the default 0.)
> 
> Is this code still the same in your version of the script?
> What's the output of the /etc/xen/scripts/network-bridge status? It should 
> normally show the bridged devices and two layout versions of the routing 
> table. Before the fix it was trying to display "eth192.168.1.231" and found 
> that it doesn't exist, of course.

The scripting in the Xen 3.2 is completely revamped, more modular and
handles current iputils properly.

> I'm tempted to try out the Xen 3.2. Is it what can be found at 
> http://xen.org/download/? What makes me wary is this last paragraph on the 
> readme:
> 
> > After installation of the binary packages, some adjustment to the
> > bootloader (grub) configuration will probably be necessary.
> 
> What do they mean? It does not install a new kernel, doesn't it?

The package doesn't even include the linux kernel. You will use the
CentOS xen linux kernel, but everytime you install/upgrade the CentOS
xen linux kernel you need to remember to edit your grub.conf to point
the xen kernel to the xen 3.2 kernel instead of the xen 3.1 kernel
that comes bundled with the CentOS xen linux kernel package.

> Did you notice any improvements with this package? I remember you wrote 
> somewhere you can now run 32bit DomUs in a 64bit Dom0 stable. Anything 
> else? Any disadvantages?

Yes 32-bit domUs run well on 64-bit dom0s, scripting works better of
course. You have full access to all the Xen VM management using
xenstore, for example add VMs to the store with 'xm new <config>'
then completely manage them through xm, xm start <name>, xm stop,
and the VM will always be visible in 'xm list', of course if you
have libvirt installed you can also use that too if you so desire.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.