[CentOS] Need help with Linux networking interfaces and NIC bonding

Mon Oct 8 04:59:55 UTC 2018
Gordon Messmer <gordon.messmer at gmail.com>

On 10/3/18 10:44 PM, Sean Son wrote:
> 1) Whenever I ping any of the devices on our network, from this server, the
> traffic goes out from the management port. I do not want the traffic to go
> out of the management port. I want it to go out through the active port of
> the NIC bond.  How do I configure the networking so that all primary
> network traffic flows to and from the NIC bonded interfaces?  I only want
> the management port to be used for SSH purposes and well, management of the
> server.

First:  You never mentioned updating /etc/iproute2/rt_tables. Did you 
create an entry there for table "t1"?  If you haven't done that, then 
your alternate routing table and your rules aren't loading.  Run "ip 
route show" and "ip route show table t1" to make sure they both exist.

Second: I agree with Anand.  You should remove the default route from 
your management interface ifcfg file, and add one to the primary 
device.  Use the rules and alternate route tables for the management 
interface *if* it needs a default route at all.  If it's only supposed 
to communicate with other devices in the management network (which is 
typical, in the systems I've managed), then it shouldn't need a default 
route at all.

Third: When you are multi-homed, you should be selecting a specific 
interface with "ping -I <interface>".  Don't rely on its automatic 
detection of the appropriate interface.

> 2) I have configured the NIC bond in active-backup mode. I notice that when
> I used another computer to do a continuous ping to the NIC bond, and then I
> disable one of the slave interfaces of the bond, the ping drops and it does
> not failover to the backup slave interface and turn  into the active one.

Can you define "disable" more specifically?

The up side of active-backup is that it should work with generic 
switches, without any specific support on their end.  The down side of 
active-backup is that your switches may remember the association between 
MAC address and port number for pretty much as long as they like, and 
some switches will take a *really* long time to update.

> Should I even use active-backup mode with the NIC bond

That depends on what component you think might fail, and what redundancy 
exists outside the system.  In my opinion: active-backup only makes 
sense if you have separate switches, since those fail more often than 
NICs.  And it only makes sense if everything behaves well when you pull 
power from one of the switches.  If you turn off one of your switches 
and the network stops working, then you shouldn't use active-backup.

> 3) Ive tested the networking, by changing  the VLAN of the NIC bonded
> ports, on the switch, to a different VLAN, and it caused the management
> port to stop responding to ping. Why is this and how do I fix that if I
> decide to one day use two different VLANs for Management and the NIC bond
> ports?

Best guess: set arp_filter to "1" and wait for your switches to update 
their MAC/port mapping