[CentOS-virt] VMs died due to hanging httpd processes

Sun Dec 12 17:41:52 UTC 2010
Jerry Franz <jfranz at freerun.com>

On 12/12/2010 06:40 AM, Dennis Jacobfeuerborn wrote:
> Monitoring show that in a timeframe of about 3 minutes the load on the
> systems shot up to over 400 before they died. Since MaxClients is set to
> 512 I suspect that the processes had a mass-lockup with each process
> constantly causing a load of 1 (similar to what happens when a process
> hangs on an NFS mount point). One of the two VMs acts as a NFS server and
> exports directories to the other VM (but doesn't mount any external NFS
> sources itself).
>
> What is strange is that both system locked up at the same time since they
> are running on two different physical hosts. The hosts run Centos 5.3 while
> the VMs run Centos 5.5 as PV Xen guests.
>
> Since the call trace looks identical on both cases I wonder if anyone has
> an idea what exactly went wrong here?

That sounds like  it might be a 'slow http' DOS attack.

http://ha.ckers.org/slowloris/

http://blog.spiderlabs.com/2010/11/advanced-topic-of-the-week-mitigating-slow-http-dos-attacks.html

-- 
Benjamin Franz