On Wed, 2013-05-01 at 16:05 -0400, Michael Mol wrote:
> I'm attempting to configure source-specific routing so that my servers
> can exist on multiple subnets from multiple upstream providers.
Kinda curious why you are attempting this without getting involved in
dynamic routing (BGP)... It's usually someone trying to do multihoming
or multi-link load balancing on the cheap without involving their ISPs
(which tends to be expensive as soon as you're talking with them about
redundant / backup loops, provider independent addresses, and BGP
peering). Generally equates to "champagne taste on a beer budget" but
there are exceptions and reasons, as I know from personal experience.
It often doesn't end well and is unreliable as network conditions
change. But that depends on your requirements and application. I'm not
one to judge - just pointing out the pitfalls.
I have done this a number of times in the past (mostly for VPN's and
redundant load-balancing links). You're probably going to have get real
down and dirty into policy routing rules and tables with iproute2. I
don't honestly believe you will be able to pull it off with the basic
stuff provided in the ifcfg-*, route-*, or static-route files (proviso
below).
I had to do it using completely custom files utilizing "ip rule" and "ip
route {add|delete} table [n]" subcommands to "ip" to build custom
matching rules and mapping them to different routing tables containing
different routes and priorities. In some cases, with OpenVPN VPNs, I
also had to incorporate iptables filtering commands to mark and match
packets and interact with the ip rule tables but I doubt you're going
that deep.
man ip-rule
--
In some circumstances we want to route packets differently depending
not only on destination addresses, but also on other packet fields:
source address, IP protocol, transport protocol ports or even packet
payload. This task is called 'policy routing'.
To solve this task, the conventional destination based routing table,
ordered according to the longest match rule, is replaced with a 'rout‐
ing policy database' (or RPDB), which selects routes by executing some
set of rules.
--
This is way beyond what the network-scripts were designed for and WAY
WAY beyond NetworkMangler's capabilities. It can be done but it's
difficult to get right and very dependent on your exact peculiar
configuration. It can be real twitchy to make work right and real
difficult to debug when it's not working right. It's also prone to
asymmetrical / triangular routes that can break things if you don't get
it right. It also will not behave robustly if one link or another goes
down (no automatic failover).
It's beyond my ability to tell you generically how to make it work (I
can't even find those old config files I use to use for multiple ISDN
links years ago). The devil is in the (your) details and here there be
dragons.
It will end up looking something like this (and these addresses are in
my real address blocks...):
ip addr add 130.205.16.16/24 dev eth0
ip addr add 130.205.17.16/24 dev eth0
ip route add table 16 default via 130.205.16.1
ip route add table 17 default via 130.205.17.1
ip rule add from 130.205.16.0/24 table 16
ip rule add from 130.205.17.0/24 table 17
Those are just from memory and may not be perfectly accurate. I haven't
needed to do any of this in years. It's just possible that the above
commands could be incorporated into the route-* files. Last time I
tried this was before iproute2 was fully integrated and I could only use
the static-routes file, which used the "route" command and would NOT
work. Since the route-* files use the ip (iproute2) command, you should
be able to integrate the appropriate rule and route add commands into
those files, but I've not done it that way so I can't really vouch for
it.
You can then see your match rules with this command:
[root at canyon ~]# ip rule ls
0: from all lookup local
32764: from 130.205.17.0/24 lookup 17
32765: from 130.205.16.0/24 lookup 16
32766: from all lookup main
32767: from all lookup default
And your route tables (including my default) like this:
[root at complex ~]# ip route ls table 16
default via 130.205.16.1 dev eth0
[root at complex ~]# ip route ls table 17
default via 130.205.17.1 dev eth0
[root at complex ~]# ip route ls
default via 99.104.36.1 dev eth2
99.104.36.0/22 dev eth2 proto kernel scope link src 99.104.38.187
130.205.16.0/24 dev eth0 proto kernel scope link src 130.205.16.16
130.205.17.0/24 dev eth0 proto kernel scope link src 130.205.17.16
130.205.38.0/24 dev br0 proto kernel scope link src 130.205.38.1
169.254.0.0/16 dev eth2 scope link metric 1003
169.254.0.0/16 dev br0 scope link metric 1007
There MAY be other ways of doing it but this was what use to work for
me, when I needed it many ages ago.
Regards,
Mike
> A rough diagram of the network layout:
> ISP1 router (blackbox, routes subnet A, address on subnet A)
> \
> -----------eth0(firewall)eth1---((servers))
> /
> ISP2 router (blackbox, routes subnet B, address on subnet B)
>
> The aim is to allow the servers to use both subnet A and subnet B. To
> allow this, any machine on both subnets must have source-specific
> routing configured, else packets originating from one ISP's AS will be
> directed at the other's router, and neither ISP cares for that.
>
> At the moment, I'm focusing on getting the second ISP properly added to
> the firewall box. The firewall box is using CentOS 6.4, and normally
> passes traffic back and forth via proxy_arp. None of my interfaces are
> NM_CONTROLLED, and NetworkManager is not installed, much less started.
>
> I've created a route-eth0:1 file that looks roughly like this:
>
> 10.0.0.1 dev eth0:1 \
> src 10.0.0.2 \
> from 10.0.0.0/29
>
> default via 10.0.0.1 dev eth0:1 \
> src 10.0.0.2 \
> from 10.0.0.0/29
>
> (Treat indented lines as continuations of the previous line)
> (No, the ISPs aren't giving me RFC1918 addresses; these are redacted.)
>
> If I run "ifup eth0:1", "ip route show" includes the lines:
>
> 10.0.0.1 dev eth0 scope link src 10.0.0.2
> 10.0.0.0/29 dev eth0 proto kernel scope link src 10.0.0.2
> default via 10.0.0.1 dev eth0
>
>
> Note that the "from 10.0.0.0/29" clause is missing. With the addition of
> a second default route on my firewall/gateway without any restriction on
> which traffic should go that way, my whole network, of course, tanks.
>
> I'm surprised it's been such a pain; I would have expected it to be a
> relatively common configuration. What's the proper way of doing
> source-specific routing on CentOS?
>
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.centos.org/pipermail/centos/attachments/20130501/3a240ea9/attachment-0005.sig>