Aleksandar Milivojevic alex@milivojevic.org wrote:
What I found with an old(er) 3ware 7500-8 (does not use same device driver as 9xxx cards) in RAID5 configuration was that it makes big difference using ext2 or ext3 (doubles the write speed, no effect on read speed).
Of course.
3Ware pairs its 64-bit ASIC in the 7000+ series with 1-4MiB of 0 wait state SRAM (Static RAM). That gives you the utmost in non-blocking JBOD, RAID-0, 1 and 10 performance. That's won't cache much more than a few (standard) 32KiB blocks -- definitely not ideal for any multi-staged writes (such as journaling).
The 9500S adds 128+MiB of multi-wait state SDRAM (Synchronous DRAM) which can buffer a lot more.
The 9550SX actually now splits the design into the legacy, non-blocking ASIC+SRAM plus a new embedded PowerPC 400 series with its own 128+MiB of DDR2 SDRAM for the ultimate in a buffering controller card. When RAID-5 is used, or extensive buffering is needed, the 64-bit ASIC (which is also the bus arbitrator) switches the incoming stream into the SDRAM which is then serviced by the embedded PowerPC 400 series.
With ext3 I used internal journal (external migh have helped, but haven't tested it). Changing journaling options and/or journal size had almost no effect. Anyhow, journaling (using default options, internal journal) should not have that high impact on write speed (not even close).
Again, considering the fact that the 7000/8000 series have an extremely small -- only 1-4MiB -- "0 wait state cache" instead of a much larger amount of "multi-wait state SDRAM buffer," this is not unexplained. 3Ware 7000/8000 series want to stream sequential writes -- especially when it comes to RAID-5. If not, it stalls.
The card was considerably faster with 2.4 kernel than with 2.6 kernel (tests run on same hardware, same configuration, ext3 file system). About 20% faster writes and 40% faster reads.
Hit 3Ware's site on optimizing the kernel 2.6 settings for the card.
And be sure to get the latest firmware for the 9500S -- that makes all the difference!
The 7000/8000 series firmware has been mature for years at 7.7.1 last time I checked.
If the new 9550SX is any suggestion, the 64-bit ASIC design is just not going to cut it at RAID-5 writes versus a full microcontroller.
AS a result, I can't recommend the 9500S. The verdict is still out on the 9550SX. But there is much promise thanx to AMCC. They _know_ the embedded PowerPC 400 series in and out.
Tom's Hardware Review just did a recent I/O queuing comparison, not actually benchmarks or CPU-interconnect load comparisons. It was rather limited in anything, although the embedded PowerPC-based 9550SX challenged the new X-Scale based Aerca's to keep up (and the X-Scale based LSI 300-8X wasn't exactly as good).