Right in the middle of doing something important on my Ubuntu box, the web quit working. However, I could get to sites in my /etc/hosts file.

Sure enough, all three of my ISPs nameservers were down.

I'm not a DNS guy, but on my CentOS boxes I always installed a caching DNS out of the box since it was part of the obvious install options. So I set the DNS to one in my Ubuntu box, and of course the port was closed.

So I ssh'd on in, opened 53 in the firewall, and the port was still closed. Tried running Firefox from the CentOS box via X forwarding over ssh but oddly, that seemed to lauch local Ubuntu firefox which didn't work. All of my CentOS boxes are headless, so all my boxes with working nameserver resolution thus only had text internet, which I haven't used in years.

locate named.conf only showed some file in dbus that I knew wasn't right, but I was able to use links and find what I needed on google, the caching nameserver conf file has a slightly different filename than standard bind named.conf.

I followed the warning, installed the system config utility for modifying it, but damn - for someone who doesn't know bind, that tool is scary looking, intimidating, I was afraid I was going to break my working caching nameserver if I messed around in it.

So I backed up the file and hand edited it to add the IP to listen on, restarted bind, and am back in business.

This is the second time in the last 6 months that all three of my ISP's nameservers have gone down, I wonder if they really are the same physical box, maybe even same network interface. What's the point of them having three if they all go down at same time?

Ah well, I don't need them anymore for that. Thankfully the caching nameservers were already installed on my CentOS boxes, two of them now listens on their external interface, never again will I use my ISPs nameserver.