[CentOS] Tracking down whats causing a high load?

[Chris Mauritz]

Ian mu wrote:
> Hiya,
>  
> Currently running Centos 4.2 x86_64 dist on a dual 3G xeon, 2G ram, 
> scsi setupand everythings been running fine on it for some time. Then 
> at 4am last night something kicked in (have mrtg running monitoring 
> when) and since then its been running a load of about 1.5 (normally 
> around 0.4). CPU usage is Cpu(s):  1.1% us,  0.6% sy,  0.0% ni, 97.9% 
> id,  0.2% wa,  0.1% hi,  0.1% si.
>  
> Can't see any new processes that would cause the load, just wondering 
> is there any way to try and track down whats actually causing this? 
> It's not excessive load, but want to add some new services and wary 
> now, its something that seems wrong given the sudden increase at 4am 
> (think thats when some o.s housekeeping tasks are normally scheduled, 
> but there's none running that I can see that started today).
>  
> Just hoping someone may have some tips on checking whats always 
> waiting or how to isolate whats happening. As said, ps -ef shows no 
> new processes, and cpu usage is very low.
>  

Have you been up to date with patches?  Have you tried running rkhunter 
and chkrootkit to see if you've been burgled?  One of the first things a 
rootkit does is replace things like ps so it's processes become "invisible."

Cheers,