Maybe somewhat of a vague question, but what would constitute a moderate and a heavy load as depicted by "top" on any particular machine? I am trying to justify running this machine with near 100% cpu utilization on both, with load averages between 5 and 5 according to top. I don't particularly see any significant slowdown in the running processes, but there is a chance the disk I/O might become an issue if too many things are going on at once. As for actual running processes, it varies between 5 and 9, depending on time of day. Memory is not really an issue, even tho usage tops out around 1.3 - 1.4Gb, and only a small usage =< 500k swap.
X does seem to get pretty sluggish at those load levels, and I've thought about dumping back to a text console for most of the runtime, but I like my Xterms and monitoring applets.
Sam
You need to check out whether the system is waiting on IO, on the version of top on Centos 4.2 it doesn't show IO wait on the display, but on the RH enterprise shipping version it does.
A load average of 9 is getting high, you expect would services like sendmail to stop listening once the system load average gets to 12.
You could check your disk performance with hdparm -t /dev/.... to see if your disks are giving you high through put. If DMA is not enabled on the disks, this can slow the system performance down and raise the load average.
P.
Sam Drinkard wrote:
Maybe somewhat of a vague question, but what would constitute a moderate and a heavy load as depicted by "top" on any particular machine? I am trying to justify running this machine with near 100% cpu utilization on both, with load averages between 5 and 5 according to top. I don't particularly see any significant slowdown in the running processes, but there is a chance the disk I/O might become an issue if too many things are going on at once. As for actual running processes, it varies between 5 and 9, depending on time of day. Memory is not really an issue, even tho usage tops out around 1.3 - 1.4Gb, and only a small usage =< 500k swap.
X does seem to get pretty sluggish at those load levels, and I've thought about dumping back to a text console for most of the runtime, but I like my Xterms and monitoring applets.
Sam
This message has been scanned for viruses and dangerous content by the *Farrows.org* http://www.farrows.org/ system scanner, and is believed to be clean.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Wednesday 01 March 2006 13:45, Peter Farrow wrote:
You need to check out whether the system is waiting on IO, on the version of top on Centos 4.2 it doesn't show IO wait on the display, but on the RH enterprise shipping version it does.
A load average of 9 is getting high, you expect would services like sendmail to stop listening once the system load average gets to 12.
As a data point, on my Sun E6500 during load testing a few months back, under Aurora SPARC Linux (I would expect similar performance from CentOS SPARC) I was pulling a load average of 250+ with little interactive degradation (command line mode). The E6500 had 14 CPU's and 16GB of RAM at the time, and was serving an ab load (apache bench) of 256 concurrent requests to a Koha integrated library system backend, over a total of 2.5 million requests. Every page hit the database at least twice, from Perl. System at that load average was serving 6 pages per second; at a concurrency of 1, system served 4 pages per second, so performance increased as load did. I would have hit it with more concurrency, but httpd was compiled with a 256 connection max limit.
On Wed, 2006-03-01 at 16:34 -0500, Lamar Owen wrote:
On Wednesday 01 March 2006 13:45, Peter Farrow wrote:
You need to check out whether the system is waiting on IO, on the version of top on Centos 4.2 it doesn't show IO wait on the display, but on the RH enterprise shipping version it does.
A load average of 9 is getting high, you expect would services like sendmail to stop listening once the system load average gets to 12.
I generally try to keep max load to no more than 2 - 2.5 times the number of processors in the machine ... so that is 8-10 for a quad processor machine ... 4-5 for a dual processor and 2-3 for a single processor machine.
That assumes that there is actual IO and load on the processes and not a high load number with lots of idle time. So, those numbers and ~100% CPU utilization is what I shoot for as maximums.
I'm sure other people have different goal numbers.
As a data point, on my Sun E6500 during load testing a few months back, under Aurora SPARC Linux (I would expect similar performance from CentOS SPARC) I was pulling a load average of 250+ with little interactive degradation (command line mode). The E6500 had 14 CPU's and 16GB of RAM at the time, and was serving an ab load (apache bench) of 256 concurrent requests to a Koha integrated library system backend, over a total of 2.5 million requests. Every page hit the database at least twice, from Perl. System at that load average was serving 6 pages per second; at a concurrency of 1, system served 4 pages per second, so performance increased as load did. I would have hit it with more concurrency, but httpd was compiled with a 256 connection max limit.
That works out at a load average of about 18 per cpu, which is of course workable as you point out, however stuff like sendmail would bulk when it reaches 12.
P.
Lamar Owen wrote:
On Wednesday 01 March 2006 13:45, Peter Farrow wrote:
You need to check out whether the system is waiting on IO, on the version of top on Centos 4.2 it doesn't show IO wait on the display, but on the RH enterprise shipping version it does.
A load average of 9 is getting high, you expect would services like sendmail to stop listening once the system load average gets to 12.
As a data point, on my Sun E6500 during load testing a few months back, under Aurora SPARC Linux (I would expect similar performance from CentOS SPARC) I was pulling a load average of 250+ with little interactive degradation (command line mode). The E6500 had 14 CPU's and 16GB of RAM at the time, and was serving an ab load (apache bench) of 256 concurrent requests to a Koha integrated library system backend, over a total of 2.5 million requests. Every page hit the database at least twice, from Perl. System at that load average was serving 6 pages per second; at a concurrency of 1, system served 4 pages per second, so performance increased as load did. I would have hit it with more concurrency, but httpd was compiled with a 256 connection max limit.
On Wednesday 01 March 2006 13:34, Lamar Owen wrote:
The E6500 had 14 CPU's and 16GB of RAM at the time, and was serving an ab load (apache bench) of 256 concurrent requests to a Koha integrated library system backend, over a total of 2.5 million requests.
But isn't load simply added by the number of processors?
EG: A load average of "1.00" on a 4-way system actually indicates 25% usage? 250/14 could indicate an effective load average of 17, which is still high, but still - how much of that load was due to I/O?
-Ben
On 3/2/06, Benjamin Smith lists@benjamindsmith.com wrote:
On Wednesday 01 March 2006 13:34, Lamar Owen wrote:
The E6500 had 14 CPU's and 16GB of RAM at the time, and was serving an ab load (apache bench) of 256 concurrent requests to a Koha integrated library system backend, over a total of 2.5 million requests.
But isn't load simply added by the number of processors?
EG: A load average of "1.00" on a 4-way system actually indicates 25% usage? 250/14 could indicate an effective load average of 17, which is still high, but still - how much of that load was due to I/O?
No load average is not just CPU but reflects all parameters. For pure CPU usage system monitor or similar tool will give better understanding. Also the processes (or dameons) running will really determine the expected load average. But as some else said for genera usage 2~3 times x number of processors is a thumb rule guide. -- Sudev Barar Learning Linux
Sudev Barar wrote:
No load average is not just CPU but reflects all parameters. For pure CPU usage system monitor or similar tool will give better understanding. Also the processes (or dameons) running will really determine the expected load average. But as some else said for genera usage 2~3 times x number of processors is a thumb rule guide. --
I don't think so. I believe, but could be wrong, that load average is the number of processes waiting to execute. Nothing else. However, I reserve the right to be wrong.
On Wed, 2006-03-01 at 19:18, Chris Mason (Lists) wrote:
Sudev Barar wrote:
No load average is not just CPU but reflects all parameters. For pure CPU usage system monitor or similar tool will give better understanding. Also the processes (or dameons) running will really determine the expected load average. But as some else said for genera usage 2~3 times x number of processors is a thumb rule guide. --
I don't think so. I believe, but could be wrong, that load average is the number of processes waiting to execute. Nothing else. However, I reserve the right to be wrong.
It is supposed to be the number of runnable processes (i.e. not waiting on i/o completion) so 1 per processor is busy, higher means something is waiting for CPU. Most programs other than graphics and number-crunching tend to wait more for i/o than CPU, so a load average of 2-3 (x processors) may not reflect noticeable delays.
On Wed, 2006-03-01 at 20:49 -0600, Les Mikesell wrote:
On Wed, 2006-03-01 at 19:18, Chris Mason (Lists) wrote:
Sudev Barar wrote:
No load average is not just CPU but reflects all parameters. For pure CPU usage system monitor or similar tool will give better understanding. Also the processes (or dameons) running will really determine the expected load average. But as some else said for genera usage 2~3 times x number of processors is a thumb rule guide. --
I don't think so. I believe, but could be wrong, that load average is the number of processes waiting to execute. Nothing else. However, I reserve the right to be wrong.
It is supposed to be the number of runnable processes (i.e. not waiting on i/o completion) so 1 per processor is busy, higher means something is waiting for CPU. Most programs other than graphics and number-crunching tend to wait more for i/o than CPU, so a load average of 2-3 (x processors) may not reflect noticeable delays.
All of you who have responded have given me good information, and I appreciate it. Altho I don't perceive this machine to *have* a load problem, I was more curious than anything as to what would constitute heavy or moderate loads. This evening, while there were processes active, I did a few tests, one of which was the hdparm -t, and I was quite shocked to see how low the thruput really was, albeit cached. Results were like 18mb/sec where as a few other times during a lighter loading, I/O was what I consider respectable, at 160 +/- mb/ 3 seconds., or approximately 54mb/sec. I also tried a few commands to see what kind of sluggishness was evident, and about the only thing I could really tell a big difference in was deleting messages from evolution, and of all things, logging out. System was *very* slow to log me out, but was about normal when I logged back in. At that time, 7 processes were running, with a load average according to top of 5.4, with 1.4gb memory in use, and 700k of swap, but not actively swapping out. There were a lot of processes that were swapped out, and only a few as runnable. Since the machine only has a single 200gb disk, I suspect part of the sluggishness comes from a bottlenecked disk I/O, just based on so many processes actually in swap.
Again, thanks for everyone's input, and thoughts. I have again, learned a thing or 3!
Regards,
Sam
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Heavy quotation editing ahead :)
On Wed, Mar 01, 2006 at 11:05:59PM -0500, Sam Drinkard wrote:
On Wed, 2006-03-01 at 20:49 -0600, Les Mikesell wrote: I was more curious than anything as to what would constitute heavy or moderate loads.
I really depends.
The load itself is only part of the equation. The process scheduler is also an important part. I once tested Riel's "fair scheduler" on a single processor P2 machine, with a nice fork bomb running. As long as I was using a different user than the one running the fork bomb, I would barely notice any problems (load was 250+).
On the other hand, I have seen places with a single high I/O process would put a 4-way system on its knees.
It is important to notice that a Sparc system would not get all locked up due to IRQs the say a PC does. That might explain why the load on that system was so high and still usable.
Bottom line, machine load average is good data, but still only part of the equation. Having a good process scheduler can make a huge difference, but be sure to have the one that is more adequate to your use, if you decide to play with it.
This evening, while there were processes active, I did a few tests, one of which was the hdparm -t, and I was quite shocked to see how low the thruput really was, albeit cached. Results were like 18mb/sec where as a few other times during a lighter loading, I/O was what I consider respectable, at 160 +/- mb/ 3 seconds., or approximately 54mb/sec.
That is I/O, all right.
System was *very* slow to log me out,
Probably cause it was saving configurations and such.
but was about normal when I logged back in.
Mostly cached and pre-loaded.
Since the machine only has a single 200gb disk, I suspect part of the sluggishness comes from a bottlenecked disk I/O, just based on so many processes actually in swap.
You can always use "iostat" to see what is happening.
- -- Rodrigo Barbosa rodrigob@suespammers.org "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
-
Benjamin Smith -
Chris Mason (Lists) -
Johnny Hughes -
Lamar Owen -
Les Mikesell -
Peter Farrow -
Rodrigo Barbosa -
Sam Drinkard -
Sudev Barar