I've been reading that it's possible to set up a system with multiple NIC to provide redundant internet connectivity such that it will switch to a secondary connection if the primary ISP fails.
Is it possible in a similar way to setup redundant LAN routing? I read that it is possible to aggregate/bond multiple NIC to stackable switches that support link aggregation and redundancy. But if only simple switches are available, is something like this possible?
e.g. System A eth0 -> lan switch/router 1 eth1 -> lan switch/router 2
System B eth0 -> lan switch 1 eth1 -> lan switch 2
Then somehow specify that, if lan switch 1 fails, the two systems will switch to using switch 2 so that in case of a switch failure, the network continues to remain operational.
On Sat, Jul 10, 2010 at 5:21 PM, Emmanuel Noobadmin centos.admin@gmail.com wrote:
I've been reading that it's possible to set up a system with multiple NIC to provide redundant internet connectivity such that it will switch to a secondary connection if the primary ISP fails.
Is it possible in a similar way to setup redundant LAN routing? I read that it is possible to aggregate/bond multiple NIC to stackable switches that support link aggregation and redundancy. But if only simple switches are available, is something like this possible?
e.g. System A eth0 -> lan switch/router 1 eth1 -> lan switch/router 2
System B eth0 -> lan switch 1 eth1 -> lan switch 2
Then somehow specify that, if lan switch 1 fails, the two systems will switch to using switch 2 so that in case of a switch failure, the network continues to remain operational. _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
I'd think for this to be possible you will need a router with multiple WAN addresses/interfaces... I am not sure how that pertains to your LAN per se.
Boris.
On Sun, Jul 11, 2010 at 05:21:50AM +0800, Emmanuel Noobadmin wrote:
e.g. System A eth0 -> lan switch/router 1 eth1 -> lan switch/router 2
System B eth0 -> lan switch 1 eth1 -> lan switch 2
Then somehow specify that, if lan switch 1 fails, the two systems will switch to using switch 2 so that in case of a switch failure, the network continues to remain operational.
If you're clever with scripting and iproute2 commands, rules and multiple routing tables, and everything's Linux, this is certainly doable. You could have your System A ping System B's IP via eth0 every minute, and on failure reassign its default route and IP to eth1. Meanwhile you can set up rules and routes on System B so that whichever NIC traffic comes in on, the response will use the same NIC ... stuff you'll find if you google around for how to be dual-homed between ISPs is quite applicable here.
It's too complex to work it out for you in detail without spending an hour on it. But I've done this sort of thing and had it work very well.
Whit
Greetings,
On 7/11/10, Emmanuel Noobadmin centos.admin@gmail.com wrote:
I've been reading that it's possible to set up a system with multiple NIC to provide redundant internet connectivity such that it will switch to a secondary connection if the primary ISP fails.
Is it possible in a similar way to setup redundant LAN routing? I read that it is possible to aggregate/bond multiple NIC to stackable switches that support link aggregation and redundancy. But if only simple switches are available, is something like this possible?
e.g. System A eth0 -> lan switch/router 1 eth1 -> lan switch/router 2
System B eth0 -> lan switch 1 eth1 -> lan switch 2
Then somehow specify that, if lan switch 1 fails, the two systems will switch to using switch 2 so that in case of a switch failure, the network continues to remain operational.
hmm.. lartc.org comes to mind to begin with...
duh.. that was too primitive. pfSense perhaps...
But then there is untangle if you want to pay them..
etc. etc.
Regards,
Rajagopal
On 7/10/2010 2:21 PM, Emmanuel Noobadmin wrote:
I've been reading that it's possible to set up a system with multiple NIC to provide redundant internet connectivity such that it will switch to a secondary connection if the primary ISP fails.
Is it possible in a similar way to setup redundant LAN routing? I read that it is possible to aggregate/bond multiple NIC to stackable switches that support link aggregation and redundancy. But if only simple switches are available, is something like this possible?
e.g. System A eth0 -> lan switch/router 1 eth1 -> lan switch/router 2
System B eth0 -> lan switch 1 eth1 -> lan switch 2
Then somehow specify that, if lan switch 1 fails, the two systems will switch to using switch 2 so that in case of a switch failure, the network continues to remain operational.
Yes. You can do it. I've done it before. All you need is the right choice of bonding mode . You set up bond0 for eth0 and eth1 and it 'just works'. To make it more robust, cross-connect the two switches as well.
Thanks for the suggestion, I'll read up more about them. The bond0 and just works sounds simple which is a Good Thing! The problem was the last time I tried to cross connect multiple switches, everything just died so there must be something a bit more involved? :D
In the mean time since my post, I came across STP (spanning tree protocol) that seems to be designed to handle this sort of thing, i.e. figure out the shortest path and prevent network shortcircuit like what I had experienced with cross connecting multiple switches.
But it apparently takes 50 seconds to reconfigure anytime sometime in the circuit fails. There is supposedly a Rapid STP that only takes 3 seconds. Several couple-of-years old search results indicate that it was tested in 2.4 kernel and will be in 2.6 kernel. However, I cannot seem to find anything newer that confirms if such functionality is really in the current kernel. Anybody has any idea?
On 7/11/10, Jerry Franz jfranz@freerun.com wrote:
On 7/10/2010 2:21 PM, Emmanuel Noobadmin wrote:
I've been reading that it's possible to set up a system with multiple NIC to provide redundant internet connectivity such that it will switch to a secondary connection if the primary ISP fails.
Is it possible in a similar way to setup redundant LAN routing? I read that it is possible to aggregate/bond multiple NIC to stackable switches that support link aggregation and redundancy. But if only simple switches are available, is something like this possible?
e.g. System A eth0 -> lan switch/router 1 eth1 -> lan switch/router 2
System B eth0 -> lan switch 1 eth1 -> lan switch 2
Then somehow specify that, if lan switch 1 fails, the two systems will switch to using switch 2 so that in case of a switch failure, the network continues to remain operational.
Yes. You can do it. I've done it before. All you need is the right choice of bonding mode . You set up bond0 for eth0 and eth1 and it 'just works'. To make it more robust, cross-connect the two switches as well.
-- Benjamin Franz _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 07/10/2010 09:48 PM, Emmanuel Noobadmin wrote:
Thanks for the suggestion, I'll read up more about them. The bond0 and just works sounds simple which is a Good Thing! The problem was the last time I tried to cross connect multiple switches, everything just died so there must be something a bit more involved? :D
Not really. You should connect the 'uplink' port to a regular port or use a cross-over cable to connect switches (assuming your switches don't auto-switch ports) and make only one connection between each switch and the next. I've got four switches chained here in my house right now without a problem to distribute my internet connection around various rooms using cheap retail 5 port d-link switches. Just don't create loops or other weird architectures, don't chain too many together, and you should be fine.
In the mean time since my post, I came across STP (spanning tree protocol) that seems to be designed to handle this sort of thing, i.e. figure out the shortest path and prevent network shortcircuit like what I had experienced with cross connecting multiple switches.
But it apparently takes 50 seconds to reconfigure anytime sometime in the circuit fails. There is supposedly a Rapid STP that only takes 3 seconds. Several couple-of-years old search results indicate that it was tested in 2.4 kernel and will be in 2.6 kernel. However, I cannot seem to find anything newer that confirms if such functionality is really in the current kernel. Anybody has any idea?
You probably don't need to worry about STP unless you are using explicitly bridging the servers' NICs. And hopefully your hardware is reliable enough that worrying about a 50 second reconfiguration is something that happens once in several years in the first place.
Emmanuel Noobadmin wrote:
Thanks for the suggestion, I'll read up more about them. The bond0 and just works sounds simple which is a Good Thing! The problem was the last time I tried to cross connect multiple switches, everything just died so there must be something a bit more involved? :D
In the mean time since my post, I came across STP (spanning tree protocol) that seems to be designed to handle this sort of thing, i.e. figure out the shortest path and prevent network shortcircuit like what I had experienced with cross connecting multiple switches.
You only really need STP when you have switches that are connected together in such a way as to have multiple paths. For the setup you first posted, you could just have two physically separate networks. That does leave the question of what solution to use to get the boxes to use the other switch if the primary one goes down. So if you connect both networks to make say a big 'circular' network, then you need STP.
But it apparently takes 50 seconds to reconfigure anytime sometime in the circuit fails. There is supposedly a Rapid STP that only takes 3 seconds. Several couple-of-years old search results indicate that it was tested in 2.4 kernel and will be in 2.6 kernel. However, I cannot seem to find anything newer that confirms if such functionality is really in the current kernel. Anybody has any idea?
You probably want to inform the switch which ports are link ports and which ports are edge ports (that is, only hosts will use the port) to reduce the amount of work and therefore time needed.
On 7/11/10, Jerry Franz jfranz@freerun.com wrote:
On 7/10/2010 2:21 PM, Emmanuel Noobadmin wrote:
I've been reading that it's possible to set up a system with multiple NIC to provide redundant internet connectivity such that it will switch to a secondary connection if the primary ISP fails.
Is it possible in a similar way to setup redundant LAN routing? I read that it is possible to aggregate/bond multiple NIC to stackable switches that support link aggregation and redundancy. But if only simple switches are available, is something like this possible?
e.g. System A eth0 -> lan switch/router 1 eth1 -> lan switch/router 2
System B eth0 -> lan switch 1 eth1 -> lan switch 2
Then somehow specify that, if lan switch 1 fails, the two systems will switch to using switch 2 so that in case of a switch failure, the network continues to remain operational.
Yes. You can do it. I've done it before. All you need is the right choice of bonding mode . You set up bond0 for eth0 and eth1 and it 'just works'. To make it more robust, cross-connect the two switches as well.
-- Benjamin Franz _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On 7/12/10, Chan Chung Hang Christopher christopher.chan@bradbury.edu.hk wrote:
You only really need STP when you have switches that are connected together in such a way as to have multiple paths. For the setup you first posted, you could just have two physically separate networks. That does leave the question of what solution to use to get the boxes to use the other switch if the primary one goes down. So if you connect both networks to make say a big 'circular' network, then you need STP.
I think I must had made a mistake in my setup example because the multipath thing seems to be what I had in mind. i.e. all machines have at least 2 paths to the others so as long as one switch is alive, functionality remains.
A more complete network diagram
WAN Managed Switch/Router -> connects to Internet, for external connectivity
Internal (for networked storage) Switch 1 (primary, 1Gbps dumb switch) -> connects to WAN Switch -> connects to Machine 1,2,3...X eth0
Switch 2 (backup, 100Mbps dumb switch) -> connects to WAN Switch -> connects to Machine 1,2,3...X eth1
Possibly a Switch 3 in the future if more data nodes are added than the primary have ports.
Idea being that the dumb switches are used solely for local data transfer between up to X number of App servers and storage nodes. The managed switch then handles only external communications as well as any firewalling.
Would connect bond0 to both switches still work without STP in this kind of a setup, or is this when STP comes in? Or is there a better network topology, given that I don't have the budget for awsome HP ProCurves ;) Reusing existing router/switch (DLink DFL-800) and dumb Gb switches.
On 07/11/2010 10:25 AM, Emmanuel Noobadmin wrote:
I think I must had made a mistake in my setup example because the multipath thing seems to be what I had in mind. i.e. all machines have at least 2 paths to the others so as long as one switch is alive, functionality remains.
A critical element of your configuration involves whether or not your managed switch links the two unmanaged switches.
If it does not, then there are two paths from any host to the other hosts and to the managed switch/router. One path will be eth0->switch 1->destination host or router. The other path will be eth1->switch 2->destination host or router. You'll need to use "active-bonding" mode in order for this to work.
If it does link them and the two are not directly linked, you have mostly the same situation. Any switch or ethernet failure will continue to allow communication between hosts, using active-bonding.
If you then link the two unmanaged switches together, there become an infinite number of paths between any two hosts. A packet can be sent from eth0->switch 1->switch 2->switch 1->...->destination. Broadcast packets *will*. That's why your network failed the last time you tried cross-connecting the switches. You can't do this with unmanaged switches. Not even with STP. STP is used when you have a mesh of managed switches to locate the shortest path between hosts and to handle the failure of a switch in the mesh.
Would connect bond0 to both switches still work without STP in this kind of a setup, or is this when STP comes in?
STP comes in when you're linking multiple managed switches in a topology other than daisy-chain or star. It has nothing to do with the configuration of network nodes which are not switches.
Idea being that the dumb switches are used solely for local data transfer between up to X number of App servers and storage nodes. The managed switch then handles only external communications as well as any firewalling.
Oh you have dumb switches in the mix? Not going to work as Gordon has said.
Would connect bond0 to both switches still work without STP in this kind of a setup, or is this when STP comes in? Or is there a better network topology, given that I don't have the budget for awsome HP ProCurves ;) Reusing existing router/switch (DLink DFL-800) and dumb Gb switches.
You don't need HP ProCurves...unless you need good jumboframe support. You will need at least D-Link 3100 switches for what you want.
-
Boris Epstein -
Chan Chung Hang Christopher -
Emmanuel Noobadmin -
Gordon Messmer -
Jerry Franz -
Rajagopal Swaminathan -
Whit Blauvelt