[CentOS] CentOS-6.7, kvm bridges, virtual interfaces, and routes

Fri Jan 8 16:16:31 UTC 2016
James B. Byrne <byrnejb at harte-lyne.ca>

I have been looking at this problem on and off for a considerable
period.  Given my lack of knowledge I have been unable to resolve this
quickly and in consequence it has been constantly shoved to the
background as other issues arise.

Here is the situation:

I have two dual-homed kvm hosts both running CentOS-6.7 and
identically configured.  These are connected to the same LAN segment
via br0/eth0 and to each other via a cross-over cable on br1/eth1. 
The IPv4 assigned to br0 on both is a publicly routeable address.  The
IPv4 assigned to br1 on both is a private address in the address space.

The guests on each host have their virtual eth0 bridged with their
host's br0 and eth1 bridged with their host's br1.  The addresses used
by the guests on eth0 are publicly routeable, the addresses used on
eth1 are private.

I would like to configure br1/eth1 on both kvm hosts such that each is
a gateway to the other.

I then also would like to configure each kvm guest of each host such
that their traffic to the private network segment on the opposite host
routes through the x-over cable via br0 whilst everything else goes
out through br1 to the LAN and gateway.

Has anyone here done anything like this? If so, can you point me to
any online resource that could more or less walk me through the
process without me having to complete the coursework for a network
engineer.  I just want to keep data transfer traffic between pairs of
kvm guests off of the public lan without having to install more
hardware. The existing cabinets are not going to support it either
space wise or power wise.

An ASCII art diagram might help, or might not.


kvmh1g1           eth0/
                  eth1/aaa.bbb.ccc.151 <-------------> |
kvmh1             br1/aaa.bbb.ccc.51                   |
            |---> br0/                     |
            X                                          |
kvmh2       |---> br0/                     |
                  br1/aaa.bbb.ccc.52                   |
kvmh2g1           eth0/                    |
                  eth1/aaa.bbb.ccc.251 <-------------> |
gateway           eth1/aaa.bbb.ccc.1 <---------------> |


I have tried multiple approaches without success and in so many
variations that I no long can clearly recall the details.

At the moment my thought is that if br0 was set to on
kvmh1 and to on kvmh2 and a routing table entry was
made on kvmh1 to send traffic addressed to through And on kvmh2 br0 was set to and a
routing table entry was made on kvmh2 to route traffic to through

I thought that if the kvm virtual guests on kvmh1 were then configured
to use addresses from while those on kvmh2 used And everything was configured to use their respective
host's br1 address as their gateway then this should work.  But I am
evidently either fundamentally wrong or I have misconfigured things

Should this set up work as I imaging?  What would be the correct
static routing table entries to make it work?

***          e-Mail is NOT a SECURE channel          ***
        Do NOT transmit sensitive data via e-Mail
James B. Byrne                mailto:ByrneJB at Harte-Lyne.ca
Harte & Lyne Limited          http://www.harte-lyne.ca
9 Brockley Drive              vox: +1 905 561 1241
Hamilton, Ontario             fax: +1 905 561 0757
Canada  L8E 3C3