[CentOS] C8 - KVM on bridge on VLAN on team issues.

Wed Jun 17 13:43:19 UTC 2020
Lamar Owen <lowen at pari.edu>

As part of my initial KVM host on C8 deployment, I decided to set up 
some HA features on the new host, specifically NIC teaming. Teaming 
seems to be bond++ of a sort, so I thought I would at least try it.  So 
here's the scenario:

1.) Server with two gigabit ethernet ports, two Cisco switches.

2.) During install, used the 'Server with GUI' group and added the 
virtualization packages.

3.) During install, set up team0 to include the two gig-e ports set up 
active-backup (two switches).

4.) During install, set up three bridges, with the slave devices being 
VLANs pointed to the team0 subinterfaces (using VLANs 68, 101, and 302; 
101 is to be the management bridge for the host, with guests on all 
three VLANs).  So, for instance, bridge101 has a slave that is VLAN101 
that points to team0.101 with a VLAN ID of 101.  The bridge101 interface 
has a manual IP address, but bridge68 and bridge302 do not (IPv4 
disabled; IPv6 Ignore)

5.) After reboot, the bridge101 interface comes up, and I successfully 
connect to the host, since the install is 8.1.1911, I ran a 'dnf update' 
up to 8.2.2004, which went well, then I successfully set up and used 
cockpit, cockpit-bridge, cockpit-machines, again over the IP address on 
bridge101.


Ok, now that the base connectivity is working:

1.) Connect to the host (traffic on bridge101 over team0.101) using 
virt-manager on my laptop and install a C8 guest, with the network 
pointed to bridge302, and a manual IP address.

2.) After reboot of guest, there is no IP connectivity to the guest's 
gateway on VLAN302.

3.) HOWEVER, the gateway's MAC address shows up in the host's bridge fdb 
for VLAN302, AND in the arp output for the guest; ALSO, the MAC address 
for the guest shows on the cisco switch 'show mac-address-table' 
output.  The output of 'ip --br link' looks normal for this 
configuration, but there's a disconnect somewhere.  So, since I see that 
VLAN101 is passing traffic to the bridge correctly (since the management 
IP is on that VLAN), I try to set up a guest on VLAN101; no dice, no 
work, but the management IP still works fine.


So, does anyone here have a working setup with KVM guests connecting to 
bridges using 802.1q VLANs on top of a team?  Or even on top of a bond 
(I can reinstall and set it up as a bond easily enough, using 
active-backup, as far as I know; and, yes, I would reinstall the host 
from scratch to do this).