I have dealt with machines that have multiple network cards in them before, but never when they were on the same subnet so this issue has never come up before.
My problem is that I can only access one IP address at a time. I started out using dhcp and found that if I went through the dhcp song-and-dance then that address became active and the other one was disabled, and vice versa.
On our local tech mailing list, a couple of the guys advised me that this is due to a routing issue and, after a bit of googling around I now understand why that is.
However, I have so far been unable to fix it.
I got rid of dhcp and set up static addresses using system-config-network.
This machine has three network cards in it, eth0 is 192.168.1.5 and I use that one to ssh into the box from this computer. (I really don't want to lose my ability to connect to eth0; this machine runs headless on a shelf about 7 feet above the floor and it would be quite an undertaking to dismantle it and bring it down to hook a monitor and keyboard to it again.)
The solution to this problem appears to be easier to describe than to implement, at least for me. I need to have each network card reply back on the same interface that it received a request from.
eth1 is 24.89.92.178 eth2 is 24.89.92.180
The gateway for both of these is 24.89.92.1
The suggestion that I got was to add two entries to the end of /etc/iproute2/rt_tables (which I did with a text editor) and run a series of ip route commands which set up a custom routing table but I'm missing something because while the custom routing tables appear to be getting set up, it's still not working.
I have studied the suggested routing commands and I think I understand what they are doing and what is supposed to be happening. But something is still missing because it's not working.
Here is what I did and what the results are. If I have missed anything let me know; this covers what I think is the relevant information as I currently understand it. I would sincerely appreciate any further advice regarding this situation. I really would like to know what I am doing wrong and also why (in the interest of learning something from this situation). It's new territory for me.
[root@audio ~]# cat /etc/iproute2/rt_tables # # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 50 access1 60 access2 [root@audio ~]# ip route add 24.89.92.0/24 dev eth1 table access1 [root@audio ~]# ip route add default via 24.89.92.1 table access1 [root@audio ~]# ip rule add from 24.89.92.178/32 lookup access1 [root@audio ~]# [root@audio ~]# ip route add 24.89.92.0/24 dev eth2 table access2 [root@audio ~]# ip route add default via 24.89.92.1 table access2 [root@audio ~]# ip rule add from 24.89.92.180/32 lookup access2 [root@audio ~]# ip route show table access2 24.89.92.0/24 dev eth2 scope link default via 24.89.92.1 dev eth1 [root@audio ~]# ip route show table access1 24.89.92.0/24 dev eth1 scope link default via 24.89.92.1 dev eth1 [root@audio ~]# ip route 24.89.92.0/24 dev eth1 proto kernel scope link src 24.89.92.178 24.89.92.0/24 dev eth2 proto kernel scope link src 24.89.92.180 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.5 169.254.0.0/16 dev eth2 scope link default via 24.89.92.1 dev eth1
[frankcox@mutt ~]$ ping 24.89.92.178 PING 24.89.92.178 (24.89.92.178) 56(84) bytes of data. 64 bytes from 24.89.92.178: icmp_seq=1 ttl=50 time=92.2 ms 64 bytes from 24.89.92.178: icmp_seq=2 ttl=50 time=96.2 ms 64 bytes from 24.89.92.178: icmp_seq=3 ttl=50 time=91.0 ms
--- 24.89.92.178 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 91.023/93.193/96.263/2.245 ms [frankcox@mutt ~]$ ping 24.89.92.180 PING 24.89.92.180 (24.89.92.180) 56(84) bytes of data.
--- 24.89.92.180 ping statistics --- 6 packets transmitted, 0 received, 100% packet loss, time 5000ms
Incidentally, it is my current understanding that anything that I do with an "ip route" command will go away on a reboot, therefore if I somehow screw up the routing on this box completely all I have to do is reboot it and I'll be back to what I had before. Which is not a bad thing at the moment. Once I have this nailed down should I put the "ip route" commands into /etc/rc.local? Or is there a better place?
I can offer one tiny bit of help ...
On Wed, Jan 20, 2010 at 11:27 AM, Frank Cox theatre@sasktel.net wrote:
Incidentally, it is my current understanding that anything that I do with an "ip route" command will go away on a reboot, therefore if I somehow screw up the routing on this box completely all I have to do is reboot it and I'll be back to what I had before. Which is not a bad thing at the moment. Once I have this nailed down should I put the "ip route" commands into /etc/rc.local? Or is there a better place?
man iptables-save
I can offer one tiny bit of help ...
On Wed, Jan 20, 2010 at 11:27 AM, Frank Cox theatre@sasktel.net wrote:
Incidentally, it is my current understanding that anything that I do with an "ip route" command will go away on a reboot, therefore if I somehow screw up the routing on this box completely all I have to do is reboot it and I'll be back to what I had before. Which is not a bad thing at the moment. Once I have this nailed down should I put the "ip route" commands into /etc/rc.local? Or is there a better place?
man iptables-save
/etc/sysconfig/iptables
mark
On Wed, 2010-01-20 at 11:33 -0500, Bob Beers wrote:
man iptables-save
That would dump the table to a file, but what would I do with the file after that? I imagine there is a way to feed that back into the ip command and reconfigure it, but I could do that with rc.local and avoid one step.
Again, I'm obviously missing something.
On Wed, Jan 20, 2010 at 11:53 AM, Frank Cox theatre@sasktel.net wrote:
On Wed, 2010-01-20 at 11:33 -0500, Bob Beers wrote:
man iptables-save
That would dump the table to a file, but what would I do with the file after that? I imagine there is a way to feed that back into the ip command and reconfigure it, but I could do that with rc.local and avoid one step.
Again, I'm obviously missing something.
Sorry, you're right, I'm completely off base. Forget iptables, you're asking about iproute2.
You can save your ip route commands in the /etc/sysconfig/network-scripts/ directory files for each interface route-ethX. They will then be automatically called when the interface is brought up on reboot, or with 'service network restart'.
IIANM, simple put all args after 'ip route add' as individual lines in the file(s).
I hope this is actually helpful this time.
On Wed, Jan 20, 2010 at 1:52 PM, Bob Beers bob.beers@gmail.com wrote:
You can save your ip route commands in the /etc/sysconfig/network-scripts/ directory files for each interface route-ethX. They will then be automatically called when the interface is brought up on reboot, or with 'service network restart'.
here's a link to a more thorough explanation:
http://www.cyberciti.biz/tips/configuring-static-routes-in-debian-or-red-hat-linux-systems.html
On Wed, Jan 20, 2010 at 2:20 PM, Bob Beers bob.beers@gmail.com wrote:
here's a link to a more thorough explanation:
http://www.cyberciti.biz/tips/configuring-static-routes-in-debian-or-red-hat-linux-systems.html
ok, last word from me on the subject, really,
.
On Wed, 2010-01-20 at 14:25 -0500, Bob Beers wrote:
On Wed, Jan 20, 2010 at 2:20 PM, Bob Beers bob.beers@gmail.com wrote:
here's a link to a more thorough explanation:
http://www.cyberciti.biz/tips/configuring-static-routes-in-debian-or-red-hat-linux-systems.html
ok, last word from me on the subject, really,
Looks like interesting reading. That's next on my list...
On 1/20/2010 1:41 PM, Frank Cox wrote:
On Wed, 2010-01-20 at 14:25 -0500, Bob Beers wrote:
On Wed, Jan 20, 2010 at 2:20 PM, Bob Beersbob.beers@gmail.com wrote:
here's a link to a more thorough explanation:
http://www.cyberciti.biz/tips/configuring-static-routes-in-debian-or-red-hat-linux-systems.html
ok, last word from me on the subject, really,
Looks like interesting reading. That's next on my list...
What's upstream? Two dsl lines from the same provider? Can you get them provisioned on different subnets?
On Wed, 2010-01-20 at 16:05 -0600, Les Mikesell wrote:
What's upstream? Two dsl lines from the same provider?
Cable, actually.
Can you get them provisioned on different subnets?
If I really had to I probably could; I have another modem in this same building from them that I've had for a while and it's on a different subnet, though everything comes in off of the same pole in the alley. These latest two got their own separate wire pulled back to the pole hookup, though, so I'm not entirely sure of what the ramification of that is.
It seems to be a moot point now, though, as the routing solution I posted earlier appears to be working wonderfully.
Incidentally, it is my current understanding that anything that I do with an "ip route" command will go away on a reboot, therefore if I somehow screw up the routing on this box completely all I have to do is reboot it and I'll be back to what I had before. Which is not a bad thing at the moment. Once I have this nailed down should I put the "ip route" commands into /etc/rc.local? Or is there a better place?
Use /etc/sysconfig/network/route-<interface> file(s).
Bob Beers wrote on Wed, 20 Jan 2010 11:33:35 -0500:
man iptables-save
this won't save the routing table
Kai
Frank Cox wrote on Wed, 20 Jan 2010 10:27:29 -0600:
I got rid of dhcp and set up static addresses using system-config-network.
Can't help you on the routing "back" issue. Just wanted to remind you that you can assign static IP addresses via DHCP to specific MAC addresses. That might be easier to maintain than assign static addresses on each machine locally.
Kai
On Wed, 2010-01-20 at 10:27 -0600, Frank Cox wrote:
My problem is that I can only access one IP address at a time. I started out using dhcp and found that if I went through the dhcp song-and-dance then that address became active and the other one was disabled, and vice versa.
I'm starting to wonder if the simplest solution to this is to punt.
If I put a $40 router between eth2 and the big scary world, then eth2 could become 192.168.whatever.whatever, and then this routing issue would go away on its own and it could still talk to the outside world (and vice versa) on its IP address from Access.
I assume, based on the fact that I have never encountered this before on machines with multiple ethernet cards that were on different subnets.
Or would this still not work as it should?
On 1/20/2010 11:31 AM, Frank Cox wrote:
On Wed, 2010-01-20 at 10:27 -0600, Frank Cox wrote:
My problem is that I can only access one IP address at a time. I started out using dhcp and found that if I went through the dhcp song-and-dance then that address became active and the other one was disabled, and vice versa.
I'm starting to wonder if the simplest solution to this is to punt.
If I put a $40 router between eth2 and the big scary world, then eth2 could become 192.168.whatever.whatever, and then this routing issue would go away on its own and it could still talk to the outside world (and vice versa) on its IP address from Access.
I assume, based on the fact that I have never encountered this before on machines with multiple ethernet cards that were on different subnets.
Or would this still not work as it should?
Why did you want this arrangement in the first place? IP routes are normally asymmetrical by design (it's a feature). I thought you said you already had a private address on eth0. Why do you need to distinguish between eth1/eth2 on the same subnet on the public side?
On Wed, 2010-01-20 at 11:48 -0600, Les Mikesell wrote:
Why did you want this arrangement in the first place? IP routes are normally asymmetrical by design (it's a feature). I thought you said you already had a private address on eth0. Why do you need to distinguish between eth1/eth2 on the same subnet on the public side?
This is a server with limited upload bandwidth. By having two outbound connections, I can use a round robin dns entry to share the load between the two connections and increase my capacity.
I'm starting to wonder if the simplest solution to this is to punt.
If I put a $40 router between eth2 and the big scary world, then eth2 could become 192.168.whatever.whatever, and then this routing issue would go away on its own and it could still talk to the outside world (and vice versa) on its IP address from Access.
I assume, based on the fact that I have never encountered this before on machines with multiple ethernet cards that were on different subnets.
Or would this still not work as it should?
Frank,
i know this has been addressed on the list a few times recently yet i dont know if that will give you a solution.
ummm, why do the two different networks need an IP on the same subnet ?
can you just bond and bridge and have the same ip on both or ???
is this a redundancy thing?
- rh
On Wed, 2010-01-20 at 09:50 -0800, R-Elists wrote:
ummm, why do the two different networks need an IP on the same subnet ?
I have had a number of people ask me why I want this arrangement, where I have two modems on a single outbound subnet.
This is (going to be) a server with limited upload bandwidth. By having two outbound connections, I can use a round robin dns entry to share the load between the two connections and increase my capacity.
I have had a number of people ask me why I want this arrangement, where I have two modems on a single outbound subnet.
This is (going to be) a server with limited upload bandwidth. By having two outbound connections, I can use a round robin dns entry to share the load between the two connections and increase my capacity. --
Frank,
do both internet feeds come from the same ISP?
DSL? Cable?
if DSL, get a 3620 and two ADSL Wics and work with the ISP to bond the DSL's together and remove one of the server nics out of the equation.
- rh
On Wednesday 20 January 2010 13:57, Frank Cox wrote:
On Wed, 2010-01-20 at 09:50 -0800, R-Elists wrote:
ummm, why do the two different networks need an IP on the same subnet ?
I have had a number of people ask me why I want this arrangement, where I have two modems on a single outbound subnet.
This is (going to be) a server with limited upload bandwidth. By having two outbound connections, I can use a round robin dns entry to share the load between the two connections and increase my capacity.
Please be aware that DNS was not designed to do what you are doing. Yes it will do a round-robin but is not connection aware. Lose a link and you lose half of the connections even though one link is still active.
On 1/20/2010 4:08 PM, Robert Spangler wrote:
On Wednesday 20 January 2010 13:57, Frank Cox wrote:
On Wed, 2010-01-20 at 09:50 -0800, R-Elists wrote:
ummm, why do the two different networks need an IP on the same subnet ?
I have had a number of people ask me why I want this arrangement, where I have two modems on a single outbound subnet.
This is (going to be) a server with limited upload bandwidth. By having two outbound connections, I can use a round robin dns entry to share the load between the two connections and increase my capacity.
Please be aware that DNS was not designed to do what you are doing. Yes it will do a round-robin but is not connection aware. Lose a link and you lose half of the connections even though one link is still active.
That depends on how the client reacts. Browsers generally are sensible and if DNS returns multiple IP's where some don't respond, will retry the connection with the others. Most other things aren't that bright.
On Wed, 2010-01-20 at 17:08 -0500, Robert Spangler wrote:
Please be aware that DNS was not designed to do what you are doing. Yes it will do a round-robin but is not connection aware. Lose a link and you lose half of the connections even though one link is still active.
I'm aware of that, but it's a limitation I can live with for this application. The whole thing is more of a "nice to have" than anything that's actually mission critical.
Frank,
I think the best way is to create bonding on eth1-eth2 and create an alias on this bond interface. If you need to use the two interfaces in same time, you can use round robin parameter on the bonding interface.
If you need help on bonding you can use this howto : http://www.generationip.com/documentation/system-documentation/65-bonding-or...
RHCE Fabien FAYE www.generationip.com
I'm starting to wonder if the simplest solution to this is to punt.
If I put a $40 router between eth2 and the big scary world, then eth2 could become 192.168.whatever.whatever, and then this routing issue would go away on its own and it could still talk to the outside world (and vice versa) on its IP address from Access.
I assume, based on the fact that I have never encountered this before on machines with multiple ethernet cards that were on different subnets.
Or would this still not work as it should?
Frank,
i know this has been addressed on the list a few times recently yet i dont know if that will give you a solution.
ummm, why do the two different networks need an IP on the same subnet ?
can you just bond and bridge and have the same ip on both or ???
is this a redundancy thing?
- rh
_______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Wed, 2010-01-20 at 10:27 -0600, Frank Cox wrote:
My problem is that I can only access one IP address at a time. I started out using dhcp and found that if I went through the dhcp song-and-dance then that address became active and the other one was disabled, and vice versa.
The solution has been found, thanks to one of the guys on our local tech mailing list (who is a truly wonderful and extremely knowedgable and helpful person).
For the benefit of anyone else who might have occasion to do this, here is the complete solution.
My IP addresses are eth1=24.89.92.178, eth2=24.89.92.180 Gateway for both is 24.89.92.1
First, add two lines to /etc/iproute2/rt_tables, so it looks like this:
# # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 50 access1 60 access2
Then add the following routes:
ip route add 24.89.92.0/24 dev eth1 table access1 ip route add default via 24.89.92.1 dev eth1 table access1 ip rule add from 24.89.92.178/32 lookup access1
ip route add 24.89.92.0/24 dev eth2 table access2 ip route add default via 24.89.92.1 dev eth2 table access2 ip rule add from 24.89.92.180/32 lookup access2
Frank Cox wrote:
I have dealt with machines that have multiple network cards in them before, but never when they were on the same subnet so this issue has never come up before.
My problem is that I can only access one IP address at a time. I started out using dhcp and found that if I went through the dhcp song-and-dance then that address became active and the other one was disabled, and vice versa.
On our local tech mailing list, a couple of the guys advised me that this is due to a routing issue and, after a bit of googling around I now understand why that is.
However, I have so far been unable to fix it.
I got rid of dhcp and set up static addresses using system-config-network.
This machine has three network cards in it, eth0 is 192.168.1.5 and I use that one to ssh into the box from this computer. (I really don't want to lose my ability to connect to eth0; this machine runs headless on a shelf about 7 feet above the floor and it would be quite an undertaking to dismantle it and bring it down to hook a monitor and keyboard to it again.)
The solution to this problem appears to be easier to describe than to implement, at least for me. I need to have each network card reply back on the same interface that it received a request from.
eth1 is 24.89.92.178 eth2 is 24.89.92.180
The gateway for both of these is 24.89.92.1
The suggestion that I got was to add two entries to the end of /etc/iproute2/rt_tables (which I did with a text editor) and run a series of ip route commands which set up a custom routing table but I'm missing something because while the custom routing tables appear to be getting set up, it's still not working.
I have studied the suggested routing commands and I think I understand what they are doing and what is supposed to be happening. But something is still missing because it's not working.
Here is what I did and what the results are. If I have missed anything let me know; this covers what I think is the relevant information as I currently understand it. I would sincerely appreciate any further advice regarding this situation. I really would like to know what I am doing wrong and also why (in the interest of learning something from this situation). It's new territory for me.
[root@audio ~]# cat /etc/iproute2/rt_tables # # reserved values # 255 local 254 main 253 default 0 unspec # # local # #1 inr.ruhep 50 access1 60 access2 [root@audio ~]# ip route add 24.89.92.0/24 dev eth1 table access1 [root@audio ~]# ip route add default via 24.89.92.1 table access1 [root@audio ~]# ip rule add from 24.89.92.178/32 lookup access1 [root@audio ~]# [root@audio ~]# ip route add 24.89.92.0/24 dev eth2 table access2 [root@audio ~]# ip route add default via 24.89.92.1 table access2 [root@audio ~]# ip rule add from 24.89.92.180/32 lookup access2 [root@audio ~]# ip route show table access2 24.89.92.0/24 dev eth2 scope link default via 24.89.92.1 dev eth1 [root@audio ~]# ip route show table access1 24.89.92.0/24 dev eth1 scope link default via 24.89.92.1 dev eth1 [root@audio ~]# ip route 24.89.92.0/24 dev eth1 proto kernel scope link src 24.89.92.178 24.89.92.0/24 dev eth2 proto kernel scope link src 24.89.92.180 192.168.1.0/24 dev eth0 proto kernel scope link src 192.168.1.5 169.254.0.0/16 dev eth2 scope link default via 24.89.92.1 dev eth1
[frankcox@mutt ~]$ ping 24.89.92.178 PING 24.89.92.178 (24.89.92.178) 56(84) bytes of data. 64 bytes from 24.89.92.178: icmp_seq=1 ttl=50 time=92.2 ms 64 bytes from 24.89.92.178: icmp_seq=2 ttl=50 time=96.2 ms 64 bytes from 24.89.92.178: icmp_seq=3 ttl=50 time=91.0 ms
--- 24.89.92.178 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 91.023/93.193/96.263/2.245 ms [frankcox@mutt ~]$ ping 24.89.92.180 PING 24.89.92.180 (24.89.92.180) 56(84) bytes of data.
--- 24.89.92.180 ping statistics --- 6 packets transmitted, 0 received, 100% packet loss, time 5000ms
Incidentally, it is my current understanding that anything that I do with an "ip route" command will go away on a reboot, therefore if I somehow screw up the routing on this box completely all I have to do is reboot it and I'll be back to what I had before. Which is not a bad thing at the moment. Once I have this nailed down should I put the "ip route" commands into /etc/rc.local? Or is there a better place?
Hi This Article should be exactly what you need
On Thu, 2010-01-21 at 10:27 +1300, Clint Dilks wrote:
This Article should be exactly what you need
That's pretty much it. I will study this some more; it's an interesting situation and I want to understand the solution.
Thanks!
-
Barry Brimer -
Bob Beers -
Clint Dilks -
fabien faye -
Frank Cox -
Kai Schaetzl -
Les Mikesell -
m.roth＠5-cent.us -
R-Elists -
Robert Spangler