[CentOS] Re: raid5 crash

Sat Jul 9 20:46:12 UTC 2005
Bryan J. Smith <b.j.smith at ieee.org>

On Sat, 2005-07-09 at 20:52 +0200, Farkas Levente wrote:
> we use this since about 3 years. that day we read all kinf of docs and 
> most people suggest to use in this way.

I know they do.  I just don't agree with it.

> faster (even you don't think so),

Oh, it probably is faster with software, because the Linux kernel will
buffer up writes far better than the small SRAM cache of the 3Ware
Escalade 8506 (only 2MB in the -8, 4MB in the -12).

The lack of DRAM buffer is why 3Ware introduced the Escalade 9500S
series -- now you get both several MBs of 0 wait state SRAM for the
switching ASIC with a good amount of DRAM buffer for RAID-5.

> easier,

Actually, I very much disagree with that assertion.  What's easier
than leaving everything -- build to failover to rebuild than to the
on-board, intelligent ASIC?  GPL drivers in the stock kernel since
2.2.15 (yes, that's _2.2_).

I'm not making this up, I have _numerous_ 3Ware Escalade 7000 series
cards that I have been deployed since Red Hat Linux 6.x / kernel 2.2.x
and have been upgraded through kernel 2.6 and I have changed _zilch_
except for maybe 1-2 firmware upgrades.  Dealing with LVM and MD changes
over the same period have been far more difficult.

I've also moved probably a dozen volumes from 6000 series to 7000 series
in my time, and even one volume set from 8000 series to 9000 series more
recently (although I have been avoiding the 9000 series because of
others with reported issues -- typical of a new 3Ware series).

> safer.

Again, totally disagree with that assertion.  I'd rather leave RAID
to a fairly static and proven firmware and driver in an intelligent,
massively queuing design, which makes the OS/software merely a dumb
block device that is hard to "screw up."  ;->

Not to trample on your issues and kick you when you are down, but didn't
you just have a problem?  ;->

The _only_ RAID-5 issue I have _ever_ had with 3Ware was when they
added it to the Escalade 6000 series.  3Ware quickly realized there
was a design consideration in the 6000 that took issue with the RAID-5
algorithm, which prompted the 7000 series design (which is also used in
the 8000 and 9000 too).

> just look trough the linux-raid list.

Well, there's several things:

1.  A lot of people throw all ATA RAID solutions into the same
bucket, and don't recognize the difference with 3Ware.

2.  Even those that do recognize 3Ware does use an on-board ASIC
intelligence don't realize how well it queues and transfers blocks
efficiently compared to traditional, "yesteryear" i960 designs
in the Promise SuperTrak and Adaptec 2400A/2800A.

3.  Some arguments I've heard say "well, I don't want the volume
to be tied to the card" when 3Ware volumes are directly movable
to newer card versions.

Yes, 3Ware cards (prior to the 9000 series) "suck" at RAID-5 writes, 
because the use a small amount of costly (transistor-wise) SRAM.  But
using SRAM also means it doesn't need battery backup either.

> we always use the latest:-)

Just wondering why you're buying 3Ware cards when you're not using the
hardware ASIC at all.

You'd be better off buying RAIDCore cards for the ATA channels if you're
going to use LVM/MD for all RAID functionality.

The only time I use LVM with 3Ware is when I'm RAID-0 striping across
two cards/volumes (on two separate PCI[-X] channels).

-- 
Bryan J. Smith                                     b.j.smith at ieee.org 
--------------------------------------------------------------------- 
It is mathematically impossible for someone who makes more than you
to be anything but richer than you.  Any tax rate that penalizes them
will also penalize you similarly (to those below you, and then below
them).  Linear algebra, let alone differential calculus or even ele-
mentary concepts of limits, is mutually exclusive with US journalism.
So forget even attempting to explain how tax cuts work.  ;->