[CentOS] Server unresponsive until reboot, memory exhausted

Thu Dec 30 09:48:47 UTC 2010
David Sommerseth <dazo at users.sourceforge.net>

On 29/12/10 17:06, cpolish at surewest.net wrote:
> On 12/28/2010 01:41 PM, james wrote:
>> You may be right about the restart, but I would like to know WHAT is
>> crashing my web server regardless. We are not running any shiftily
>> coded sites or apps on this server that I'm aware of (obviously
>> something is shifty!). Is anyone aware of any other methods for
>> drilling into the problem?
>
> You may be dealing with a request that crashes before any data
> makes it into the logs.
>
> Here's a trick I have used (posted Feb 10, 2008 by "Phantom"
> in alt.apache.configuration):
>
> ls -l `ps -C httpd h | sed -r "s/^\s*([0-9]+) .+$/\/proc\/\\1\/cwd/"`
>
> which prints out the directory that each Apache worker thread is
> accessing. If you can narrow it down to the pid, you can use lsof to
> identify the particular file being being processed. Please keep us
> posted!

If you can track down the pid, you can easily check which file descriptors 
being in used and which files or sockets they relate to by checking the 
/prod/$PID/fd directory.


kind regards,

David Sommerseth