[CentOS] load level?

Fri Apr 25 17:34:01 UTC 2008
Jason Pyeron <jpyeron at pdinc.us>

I have a server holding a 2.5-4.0 load level consistently, where should I
start debugging it?

top - 13:32:01 up 167 days,  5:45,  6 users,  load average: 2.87, 1.83, 1.51
Tasks: 111 total,   1 running, 110 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0% us,  0.0% sy,  0.2% ni, 99.8% id,  0.0% wa,  0.0% hi,  0.0% si
Mem:    895728k total,   871996k used,    23732k free,   190612k buffers
Swap:  1802232k total,      216k used,  1802016k free,    63776k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    1 root      16   0  4756  556  460 S    0  0.1   0:35.34 init
    2 root      RT   0     0    0    0 S    0  0.0   0:02.83 migration/0
    3 root      34  19     0    0    0 S    0  0.0   0:27.27 ksoftirqd/0
    4 root      RT   0     0    0    0 S    0  0.0   0:05.03 migration/1
    5 root      34  19     0    0    0 S    0  0.0   0:22.88 ksoftirqd/1
    6 root       5 -10     0    0    0 S    0  0.0   0:03.15 events/0
    7 root       5 -10     0    0    0 S    0  0.0   0:01.98 events/1
    8 root       5 -10     0    0    0 S    0  0.0   0:00.00 khelper
    9 root       5 -10     0    0    0 S    0  0.0   0:00.00 kblockd/0
   10 root       5 -10     0    0    0 S    0  0.0   0:00.00 kblockd/1
   11 root      15   0     0    0    0 S    0  0.0   0:00.00 khubd
   66 root      15   0     0    0    0 S    0  0.0   0:16.14 pdflush
   67 root      15   0     0    0    0 S    0  0.0   2:30.44 kswapd0
   68 root      15 -10     0    0    0 S    0  0.0   0:00.00 aio/0
   69 root       5 -10     0    0    0 S    0  0.0   0:00.00 aio/1
  213 root      22   0     0    0    0 S    0  0.0   0:00.00 kseriod
  450 root       5 -10     0    0    0 S    0  0.0   0:00.00 ata/0
  451 root       5 -10     0    0    0 S    0  0.0   0:00.00 ata/1
  452 root       6 -10     0    0    0 S    0  0.0   0:00.00 ata_aux
  456 root      15   0     0    0    0 S    0  0.0   0:00.00 scsi_eh_0
  457 root      15   0     0    0    0 S    0  0.0   0:00.00 scsi_eh_1
  465 root      15   0     0    0    0 S    0  0.0   0:00.00 scsi_eh_2
  466 root      15   0     0    0    0 S    0  0.0   0:00.00 scsi_eh_3
  494 root      15   0     0    0    0 S    0  0.0   4:03.36 kjournald
  542 root      16   0 24192  596  416 S    0  0.1   0:36.55 ypbind
  595 root      15   0     0    0    0 S    0  0.0   0:00.00 rpciod
  596 root      18   0     0    0    0 S    0  0.0   0:00.00 lockd
  775 apache    15   0  185m  40m 3628 S    0  4.6   0:00.59 httpd
 1832 root       5 -10     0    0    0 S    0  0.0   0:00.00 kedac
 1904 root      16   0 10828 1072  856 S    0  0.1   0:00.00 pppd
 1973 root       6 -10     0    0    0 S    0  0.0   0:00.00 kauditd
 1998 root      20   0     0    0    0 S    0  0.0   0:00.00 scsi_eh_4
 1999 root      15   0     0    0    0 S    0  0.0   3:10.72 usb-storage
 2176 root      15   0     0    0    0 S    0  0.0   0:00.01 kjournald
 3070 root      15   0  3632  552  440 S    0  0.1   1:08.11 syslogd
 3074 root      15   0  2540  372  296 S    0  0.0   0:00.04 klogd
 3087 root      16   0  2552  300  200 S    0  0.0   0:30.06 irqbalance
 3155 root      15   0  2916  488  256 S    0  0.1   0:00.97 smartd
 3215 root      15   0  8716  776  612 S    0  0.1   0:00.00 xinetd
 3293 root      16   0 40836 2672  984 S    0  0.3   1:07.23 sendmail
 3303 smmsp     16   0 33504 2108  812 S    0  0.2   0:00.99 sendmail
 3313 root      16   0  4180  324  244 S    0  0.0   0:00.00 gpm
 3323 root      16   0  175m  35m 9064 S    0  4.0   0:13.24 httpd
 3332 root      16   0 57092  976  560 S    0  0.1   0:50.01 crond
 3353 xfs       16   0 10020 1472  756 S    0  0.2   0:00.17 xfs

-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-                                                               -
- Jason Pyeron                      PD Inc. http://www.pdinc.us -
- Principal Consultant              10 West 24th Street #100    -
- +1 (443) 269-1555 x333            Baltimore, Maryland 21218   -
-                                                               -
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
 
This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise private information. If you
have received it in error, purge the message from your system and
notify the sender immediately.  Any other use of the email by you
is prohibited.