On Wed, 2013-05-01 at 17:52 -0400, Michael Mol wrote:
> On 05/01/2013 05:15 PM, Michael H. Warfield wrote:
> > On Wed, 2013-05-01 at 16:05 -0400, Michael Mol wrote:
> >> I'm attempting to configure source-specific routing so that my servers
> >> can exist on multiple subnets from multiple upstream providers.
> >
> > Kinda curious why you are attempting this without getting involved in
> > dynamic routing (BGP)... It's usually someone trying to do multihoming
> > or multi-link load balancing on the cheap without involving their ISPs
> > (which tends to be expensive as soon as you're talking with them about
> > redundant / backup loops, provider independent addresses, and BGP
> > peering). Generally equates to "champagne taste on a beer budget" but
> > there are exceptions and reasons, as I know from personal experience.
> > It often doesn't end well and is unreliable as network conditions
> > change. But that depends on your requirements and application. I'm not
> > one to judge - just pointing out the pitfalls.
>
> Yup, I know.
>
> Intent is to maintain the old, slow (but has an SLA) connection as a
> fallback, and migrate services to the new connection piecemeal.
> Meanwhile, the same DNS server on the new connection can be, e.g. "ns3".
> The same mailserver can have a new MX on the new connection...likely
> prioritized to it.
>
> Inbound services can be load-balanced fairly easily via DNS, if TTLs are
> kept low, and records updated in response to link state. It's not
> anycast DNS, but it also doesn't require to you get BGP peering and PI
> space. (I don't even know if I could *get* IPv4 PI space at this point.
> I certainly know I wouldn't be able to if I waited a year...)
>
> >
> > I have done this a number of times in the past (mostly for VPN's and
> > redundant load-balancing links). You're probably going to have get real
> > down and dirty into policy routing rules and tables with iproute2. I
> > don't honestly believe you will be able to pull it off with the basic
> > stuff provided in the ifcfg-*, route-*, or static-route files (proviso
> > below).
> >
> > I had to do it using completely custom files utilizing "ip rule" and "ip
> > route {add|delete} table [n]" subcommands to "ip" to build custom
> > matching rules and mapping them to different routing tables containing
> > different routes and priorities. In some cases, with OpenVPN VPNs, I
> > also had to incorporate iptables filtering commands to mark and match
> > packets and interact with the ip rule tables but I doubt you're going
> > that deep.
>
> Yeah, I've gone that deep. And a tad deeper. I had almost *everything*
> working by hand, and went to figure out how to convert it to idomatic
> CentOS network configuration scripts. And took my network down *three
> times* because of the script-processing stripping things out.
>
> >
> > man ip-rule
> >
> > --
> > In some circumstances we want to route packets differently depending
> > not only on destination addresses, but also on other packet fields:
> > source address, IP protocol, transport protocol ports or even packet
> > payload. This task is called 'policy routing'.
> >
> > To solve this task, the conventional destination based routing table,
> > ordered according to the longest match rule, is replaced with a 'rout‐
> > ing policy database' (or RPDB), which selects routes by executing some
> > set of rules.
> Yup. I went through LARTC before writing a line of code, just to be sure.
> Curiously, at least one guy has reported success:
> http://sysadminsjourney.com/content/2009/04/15/doing-simple-source-policy-routing-centos/
> Now, the only thing different between his setup and mine (apart from my
> using ethN:1 instead of ethN, as all three routers hang off the same
> ethernet segment) is that were his guide says:
> echo "default table CorpNet via 10.0.0.1" >
> /etc/sysconfig/network-scripts/route-eth1
Ok... Wow... If that's the only difference between his description and
what you did, you certainly left A LOT out. He's using both rules and
tables neither of which you made any mention of in your original post.
At this point, having read that article, I will eat my earlier words
(not the first time and certainly won't be the last time). I guess you
can now do this using the standard files, it's just that I haven't done
it in so long that you couldn't do it back then (my excuse and I'm
sticking with it). Following his description, I could easily reproduce
my old setups using ifcfg-ethN, rule-ethN, and route-ethN. I'm
impressed. I don't need it any more but - nice... That makes it a lot
easier than what I had to figure out.
Ok... So, I'm assuming you properly set up the rules-ethN file as well
(and the proper entry in /etc/iproute2/rt_tables? You made no mention
of that in your OP. That's a very crucial bit there.
Also, in your OP you mentioned this:
On Wed, 2013-05-01 at 16:05 -0400, Michael Mol wrote:
> I've created a route-eth0:1 file that looks roughly like this:
>
> 10.0.0.1 dev eth0:1 \
> src 10.0.0.2 \
> from 10.0.0.0/29
>
> default via 10.0.0.1 dev eth0:1 \
> src 10.0.0.2 \
> from 10.0.0.0/29
You're not showing table numbers or names there so it's not clear if you
are using different route tables or not (which you MUST do and associate
them with appropriate match rules).
According to "man ip-route" on my router the "from" stanza is not valid
in a "route add" (route-ethN files) and in a "route ls" is only
applicable to "cloned" routes. What you wrote can not literally work,
by my reading of the "ip" man pages.
You get the source matching from the "rules" not the "routes". You
haven't mentioned (or acknowledged) anything about them but they are
crucial (as are the use of multiple tables). What did you set up for
your match rules? No match rules, then only the default and local
tables are going to be used. Your "from" specifier goes in your rules,
not your routes.
When I look at my route tables, I see src associated with an appropriate
route. I don't see any "from" matches because they are not in the route
tables they're in the rules. You also have to look at "ip rules ls".
That's where your "from" is going to show up and then tell you what
table it's going to use as its routing table.
> My first pass at making my code platform-idomatic effectively was:
> echo "default via 10.0.0.1 table CorpNet" >
> /etc/sysconfig/network-scripts/route-eth1
> (the "table $table" clause in mine was at the end of the line, following
> the pattern I'd read in LARTC, rather than near the beginning of the line.)
Ok, so you are using the table named "CorpNet" which you must have added
to /etc/iproute2/rt_tables in advance (his step 1) otherwise you can
only use table numbers. The position of the "table CorpNet" should make
no difference. Have you added a corresponding match rule (his step 3)?
He defines "CorpNet" as table 200.
It looks like, if you did everything he describes in that article, it
should work. I've done this manually and what he describes there
exactly matches what I would expect to work. You really haven't given
us the complete picture of what you did (or if you did, you left out a
couple of steps). Obfuscate the addresses and names if you must but you
need to be clearer on the contents of all the configuration files.
/etc/iproute2/rt_tables
/etc/sysconfig/network-scripts/ifcfg-eth0
/etc/sysconfig/network-scripts/route-eth0
/etc/sysconfig/network-scripts/rule-eth0
I'm also not real sure how this works with additional addresses like
eth0:N. Not sure if you need these in the parent device or the alias
device. Looks like putting under the alias interface should be fine.
Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.centos.org/pipermail/centos/attachments/20130503/87fe893a/attachment-0004.sig>