[CentOS] DNS forwarding vs recursion

Mon Apr 1 13:11:01 UTC 2013
Michael H. Warfield <mhw at WittsEnd.com>

On Thu, 2013-03-28 at 11:29 -0700, John R Pierce wrote:
> On 3/28/2013 11:11 AM, Jorge Fábregas wrote:
> > On 03/28/2013 02:05 PM, John R Pierce wrote:
> >> >is it as simple as adding allow-recursion{} with  the appropriate private
> >> >subnets and localhost to named.conf ?
> > Yes.  That's basically it.

> k, thanks, looks like its working!


I'm one of the original founders of Internet Security Systems, now part
of IBM.  I was a founder of the ISS X-Force and am still one of their
"Senior Wizards" and a senior security researcher.  However!  I am not
speaking for or on behalf or representing ISS or X-Force or IBM...  I am
writing, personally, as a professional "white hat hacker" and security
researcher.  This will be a bit of a long soap-box / tub thump.


I love to believe in coincidences and I'm a firm believer in
coincidences...  But...  The timing of your original post leads me to
think otherwise.  The fact that you say "looks like it's working" tells
me that you're seeing something active on your network that makes me
believe that this is not a coincidence.

By any chance, might you have been one of the thousands of open
resolvers that got caught up in the Spamhaus / Cyberbunker debacle of
the last two weeks?  Don't bother to answer that and don't worry about
it if you were.  If you were, you were only one (or a handful) out of
thousands who were abused in this attack on Spamhaus.  They all (you
all) were, in turn, only a small fraction of the tens (or hundreds) of
thousands of open resolvers present on the net, so you're actually in
good (large) company (unfortunately).

The suggested solution will solve a (BIG) part of the problem.  It WILL
solve the problem being exploited in the Spamhaus attack, that of
abusing open resolvers in attacking others in a DNS resource
amplification attack.  Unfortunately, it does NOT solve the problem of
someone attacking you exploiting your own resolvers against you.

Resource amplification attacks like this utilize spoofed packets
(packets spoofed to be "from" the actual target).  The amplified
payloads are the replies back to the spoofed addresses.

Now, suppose someone has YOU in their crosshairs.  They identify your
resolvers (not really a major task - especially if you make the mistake
of using your authoritative name servers as resolvers as well) and then
proceed to spoof packets into your resolvers as if they were coming from
your IP addresses.  By the time the packets reach your resolvers, they
have no way to tell if the source address is legitimate or not, and will
match your recursion allow rules.  It's the the job of your security
perimeter firewalls to filter local vrs foreign packets and on-session
vrs unsolicited packets.  Consequently, your resolvers will amplify that
traffic (factor of 100 or more) and reflect it back to your internal

It's actually amusing how attackers can abuse the very nature of the DNS
for this.  They prime an authoritative name server with a MASSIVE
payload and then spoof the first request into the target resolver for
that resource.  The resolver then caches it and then subsequent requests
are served out of the resolver's cache and doesn't load down the
attacker's name servers further.  So, he can then hit you with hundreds
of thousands of spoofed requests delivering GB of data at his intended
target (possibly you) having only serviced a single query request for a
few K.  Gotta love that for efficiency.

Years ago, I actually had a customer (a major multinational corporation
who shall remain unnamed) come under such an attack and was crying for
help.  Restricting your resolvers to "local networks" does not help in
this case.  They had an exposed resolver, outside of their firewall
perimeter, and the attacker was spoofing packets from their internal
addresses.  Even if your firewall blocks those responses (stateful
firewall) it still overloads your pipe in front of the firewall.  It was
crushing them with no way to filter it.

Let me make this clear.  Any recursive name server, restricted or not,
which can be reached from the greater Internet can be abused to attack
the networks for which it is configured to allow recursion.  The
restrictions limit what can be attacked.  Without firewalling an
external filtering, it does not restrict where you can be attacked from,
since it's a spoofed attack in the first place.

Years before, I had published a series of articles for our customers on
"Robust DNS Deloyments".  One of the recommendations was to NOT have
your authoritative name servers be recursive name servers at all (except
for localhost).  That was their only solution.  They had to scramble and
shut down those external recursive resolvers entirely and switch to
internal ones protected behind their firewalls that could block the
spoofed packets.

The argument goes at this...

Your authoritative name servers (at least some of them) must be public.
That's their nature, to publicly advertise your domain name information
(exception here is that I recommend the master name server be private
and all your public name servers be slaves off the private master - but
that's a deeper story).  That implies that outside, foreign, addresses
must be able to communicate with your public authoritative name servers.
Foreign addresses must be able to initiate queries with your
authoritative name servers in order to ask for the information they are

Not so with your resolvers.  Your resolvers initiate communications with
other name servers but no foreign name server should ever initiate a
communication with your resolver (it's not authoritative for anything).
Your resolvers should be maintained behind firewalls that can, in turn,
maintain a knowledge of what constitutes "local" addresses on one side
and "foreign" addresses on the other side.  Firewall rules then prevent
packets from the foreign side spoofed to look like they came from
addresses on the local side.  THAT prevents anyone from abusing your
resolvers to attack both others and you.

The net (excuse the pun) gist of it is that resolvers and authoritative
name servers provide their services in different security zones and
should not be mixed on the same server.  As easy as it is to provide
both services on the same name servers, it is not best common practice
to do so.  Provide those services in their respective zones on separate
systems, appropriately protected and locked down.

If you absolutely MUST combine them (and I would love to hear the
rational as to why, beyond cost and laziness) then, by all means,
restrict recursion to your local networks, with the understanding that
they can still be abused to attack yourself..

I don't know where you are in the Internet "food chain" (end consumer,
ISP, Tier 1 provider, or backbone) but if you are in the routing chain
(you manage or provide routing - anyone other than an end consumer) then
it's also very important to implement BCP (Best Common Practice) 38.
BCP 38 recommends router egress filtering.  That is, you only route out
what will route back in.  That prevents you (or any of your customers)
from being a spoofing source.  That strikes at the heart of many of
these types of attacks.

With Linux systems acting as routers, that's actually pretty easy and is
(last time I looked) enabled by default.  On Linux, this is the "route
verification" option.  With route verification enabled on Linux, it will
not send a packet on an interface if the source address would ALSO be
routed out through the same interface (i.e. spoofed).  "Triangular
routing" (not uncommon with some VPNs) is an exception to this and
should be a very rare exception and treated as a strict exception.  This
can get real complicated and difficult if you are running BGP (Border
Gateway Protocol) on overtaxed gateway routers but still should be
implemented if at all possible.

Routing issues and BCP38 aside, you really should separate your
authoritative an recursive name servers if at all possible.




> -- 
> john r pierce                                      37N 122W
> somewhere on the middle of the left coast

Michael H. Warfield (AI4NB) | (770) 985-6132 |  mhw at WittsEnd.com
   /\/\|=mhw=|\/\/          | (678) 463-0932 |  http://www.wittsend.com/mhw/
   NIC whois: MHW9          | An optimist believes we live in the best of all
 PGP Key: 0x674627FF        | possible worlds.  A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.centos.org/pipermail/centos/attachments/20130401/2ae24567/attachment-0004.sig>