[CentOS] network redundancy via two nics, two routers?

[Feizhou]

>> Should it not be that there would be a third ip that would be set as 
>> the gateway and the two routers will claim the ip as needed? So it 
>> would be the same ip used as the gateway whether the packets go out 
>> through eth0 or eth1.
> 
> Feizhou, that would be ideal... but I don't know of any way in the HP 
> ProCurve switches of doing this. Each switch would have to health-check 
> the other, and the standby would have to pick up that IP if the primary 
> went down. They don't seem to support this. That would be 100% perfect, 
> though; because in that setup, CentOS doesn't have to even know that the 
> bonded interface is split between two physical switches or that the 
> gateway is relocatable.

Oh sorry, I read your post wrong. So you have:

link1		link2
	2.2.2.1 HSRP
	VLAN1
Procurve1	Procurve2
	VLAN2
1.1.1.254	1.1.1.253
	1.1.1.2 BONDED
	ServerA

I am not sure but I think you could use iproute2 tools.

Create two routing tables. Add for example the below to 
/etc/iproute2/rt_tables:

10 switchA
11 switchB

Create a script or put the commands to populate those routing tables 
into rc.local

ip route add 1.1.1.0/24 dev eth0 table switchA
ip route add default via 1.1.1.254 dev eth0 table switchA
ip route add 1.1.1.0/24 dev eth1 table switchB
ip route add default via 1.1.1.253 dev eth1 table switchB

Below are commands that make the system lookup appropriate routing 
tables depending on interface.
ip rule add from 1.1.1.2 dev eth0 lookup switchA
ip rule add from 1.1.1.2 dev eth1 lookup switchB

The question is whether using dev ethX will be honoured...as I 
understand that you will get an interface called bond0...
> 
> So the question remains: besides installing software that health-checks 
> the connection, is there a way to configure linux bonding to use 
> different gateways based on the physical NIC in a bonded pair? (Or, a 
> way to tell it to try two different gateways; and if one is down, to try 
> the other? I.e. some sort of route cost solution?)

Or maybe you could have a regular check on the link status with mii-tool 
... if eth0 is down, use eth1 gw as default route and vice versa.