[CentOS] routing multiple network cards on a single subnet

I have dealt with machines that have multiple network cards in them
before, but never when they were on the same subnet so this issue has
never come up before.

My problem is that I can only access one IP address at a time.  I
started out using dhcp and found that if I went through the dhcp
song-and-dance then that address became active and the other one was
disabled, and vice versa.

On our local tech mailing list, a couple of the guys advised me that
this is due to a routing issue and, after a bit of googling around I now
understand why that is.

However, I have so far been unable to fix it.

I got rid of dhcp and set up static addresses using
system-config-network.

This machine has three network cards in it, eth0 is 192.168.1.5 and I
use that one to ssh into the box from this computer.  (I really don't
want to lose my ability to connect to eth0; this machine runs headless
on a shelf about 7 feet above the floor and it would be quite an
undertaking to dismantle it and bring it down to hook a monitor and
keyboard to it again.)

The solution to this problem appears to be easier to describe than to
implement, at least for me.  I need to have each network card reply back
on the same interface that it received a request from.

eth1 is 24.89.92.178
eth2 is 24.89.92.180

The gateway for both of these is 24.89.92.1

The suggestion that I got was to add two entries to the end
of /etc/iproute2/rt_tables (which I did with a text editor) and run a
series of ip route commands which set up a custom routing table but I'm
missing something because while the custom routing tables appear to be
getting set up, it's still not working.

I have studied the suggested routing commands and I think I understand
what they are doing and what is supposed to be happening.  But something
is still missing because it's not working.

Here is what I did and what the results are.  If I have missed anything
let me know; this covers what I think is the relevant information  as I
currently understand it.  I would sincerely appreciate any further
advice regarding this situation.  I really would like to know what I am
doing wrong and also why (in the interest of learning something from
this situation).  It's new territory for me.

[root at audio ~]# cat /etc/iproute2/rt_tables 
#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
#1      inr.ruhep
50      access1
60      access2
[root at audio ~]# ip route add 24.89.92.0/24 dev eth1 table access1
[root at audio ~]# ip route add default via 24.89.92.1 table access1
[root at audio ~]# ip rule add from 24.89.92.178/32 lookup access1
[root at audio ~]# 
[root at audio ~]# ip route add 24.89.92.0/24 dev eth2 table access2
[root at audio ~]# ip route add default via 24.89.92.1 table access2
[root at audio ~]# ip rule add from 24.89.92.180/32 lookup access2
[root at audio ~]# ip route show table access2
24.89.92.0/24 dev eth2  scope link 
default via 24.89.92.1 dev eth1 
[root at audio ~]# ip route show table access1
24.89.92.0/24 dev eth1  scope link 
default via 24.89.92.1 dev eth1 
[root at audio ~]# ip route
24.89.92.0/24 dev eth1  proto kernel  scope link  src 24.89.92.178 
24.89.92.0/24 dev eth2  proto kernel  scope link  src 24.89.92.180 
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.5 
169.254.0.0/16 dev eth2  scope link 
default via 24.89.92.1 dev eth1 

[frankcox at mutt ~]$ ping 24.89.92.178
PING 24.89.92.178 (24.89.92.178) 56(84) bytes of data.
64 bytes from 24.89.92.178: icmp_seq=1 ttl=50 time=92.2 ms
64 bytes from 24.89.92.178: icmp_seq=2 ttl=50 time=96.2 ms
64 bytes from 24.89.92.178: icmp_seq=3 ttl=50 time=91.0 ms

--- 24.89.92.178 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 91.023/93.193/96.263/2.245 ms
[frankcox at mutt ~]$ ping 24.89.92.180
PING 24.89.92.180 (24.89.92.180) 56(84) bytes of data.

--- 24.89.92.180 ping statistics ---
6 packets transmitted, 0 received, 100% packet loss, time 5000ms

Incidentally, it is my current understanding that anything that I do
with an "ip route" command will go away on a reboot, therefore if I
somehow screw up the routing on this box completely all I have to do is
reboot it and I'll be back to what I had before.  Which is not a bad
thing at the moment.  Once I have this nailed down should I put the "ip
route" commands into /etc/rc.local?  Or is there a better place?


-- 
MELVILLE THEATRE ~ Melville Sask ~ http://www.melvilletheatre.com