[CentOS] oops, or how to bring a datacenter router down with one setting

Fri Feb 10 20:01:02 UTC 2012
Lamar Owen <lowen at pari.edu>

On Feb 9, 2012, at 6:54 PM, Bob Hoffman wrote:
> entire ip block went out.
>
> when I called datacenter they told me the router was under attack  
> and I
> was like 'uh oh' and told them to just shut off my computer I would be
> there to fix it. They did not believe me.
> An hour later I was there and deleted the eth1 point to the br0 and  
> all
> was fine.
> Meanwhile they were all around the router trying to stop the attack.
> (it was just the router for me and others in that room....oops)
>
> I wonder if they will boot me from the center now?
> How is it possible that it did that so quickly?
> Such an easy way to bring down routers, wow, a hacker could have a  
> field
> day.

If you weren't running a spanning-tree on your Linux bridge, and their  
switch ports aren't sending you BPDU's for STP, then you found out  
what happens when you activate a bridging (from the point of view of  
the switch, not the Linux bridging) loop.  Been there, done that.   
Most monitoring tools are written to track layer-3 happenings, and  
this is happening at layer 2.  And it will take down that whole layer  
2 broadcast domain, that's for sure.

And since many, if not most, tools are working at layer 3 and dealing  
with IP flows and not actual ethernet traffic, none of the typical  
layer 3 tools will give any indication why the network just bogged  
down to a halt; you just about have to have a network probe (like  
wireshark) on a SPAN port to catch it, unless you know some of the  
telltale signs.  On a gigabit switch a fully saturating bridge loop  
can form in less than a second, and bring things close to a halt.

Most datacenter switches have configurable parameters to guard against  
loops (Cisco even has a feature called, appropriately enough,  
loopguard, but this may or may not fix this case).