[CentOS] 3Ware 9550SX and latency/system responsiveness

Tue Oct 2 13:24:14 UTC 2007
Ross S. W. Walker <rwalker at medallion.com>

matthias platzer wrote:
> 
> hello,
> 
> i saw this thread a bit late, but I had /am having the exact 
> same issues 
> on a dual-2-core-cpu opteron box with a 9550SX. (Centos 5 x86_64)
> What I did to work around them was basically switching to XFS for 
> everything except / (3ware say their cards are fast, but only on XFS) 
> AND using very low nr_requests for every blockdev on the 3ware card.
> (like 32 or 64). That will limit the iowait times for the 
> cpus and make 
> the 3ware-drives respond faster (see yourself with iostat -x 
> -m 1 while 
> benchmarking).
> If you can, you could also try _not_ putting the system disks on the 
> 3ware card, because additionally the 3ware driver/card gives writes 
> priority. People suggested the unresponsive system behaviour 
> is because 
> the cpu hanging in iowait for writing and then reading the system 
> binaries won't happen till the writes are done, so the 
> binaries should 
> be on another io path.
> 
> All this seem to be symptoms of a very complex issue consisting of 
> kernel bugs/bad drivers/... and they seem to be worst on a AMD/3ware 
> Combination.
> here is another link:
> http://bugzilla.kernel.org/show_bug.cgi?id=7372

Actually the real-real fix was to use the 'deadline' or 'noop' scheduler
with this card as the default 'cfq' scheduler was designed to work with
a single drive and not a multiple drive RAID, so it acts as a govenor on
the amount of IO that a single process can send to the device and when
you do multiple overlapping IOs performance decreases instead of
increases.

Personnally I always use 'deadline' as my scheduler of choice.

Of course if your drivers are broken that will always negatively impact
performance.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.