First of all, thanks to everybody for their responses on this thread.
On Tue, 2005-11-01 at 10:31, centos-bounces@centos.org wrote:
Aleksandar Milivojevic alex@milivojevic.org wrote:
Correct me if I'm wrong, but you got 233MB/s for reads (the block read test).
Oh, good catch! I didn't even see that when responding (I assumed he could interpret the bonnie benchmark). And if I see that correctly, that was with a 12GiB file (on a system that had 6GiB RAM).
I was fixated on the per char read rate and didn't pay much attention to the block read rate. ;-)
Assuming your disks can do 50MB/s sustained transfer rate each, you are preatty darn close to the theoretical maximum of (6 - 1) * 50MB/s = 250MB/s for 6 disk RAID5.
At least the read speeds are performing as can be reasonably expected for this particular configuration. I am certainly happy with that result.
On reads, yes. 3Ware is clearly leveraging the ASIC's non-blocking I/O for reads from RAID-5, which basically act like RAID-0.
RAID5 as such is bad choice for file systems that will have more than about 30% of writes (out of total I/O).
He still should be seeing at least 100MBps for RAID-5 writes on a 3Ware Escalade 9500S with 6-discs (180MBps is about the maximum for RAID-5 writes on the 9500S' ASIC with DRAM). The ASIC is fairly good at sequential writes to RAID-5, and there is enough DRAM to buffer all but the heaviest of random I/O.
Still, the new 9550SX series has a PowerPC. AMCC's influence is clearly being pressed on their 3Ware acquisition, as they are _the_ company for the IBM embedded PowerPC 400 line now. The 9550SX is supposed to be cable of 380MBps for RAID-5 writes -- double the 9500S best benchmarks.
If most of the I/O will be writes, and you care about performance, you should use RAID-10.
Yep, mega-dittos on that point.
Remember, writes to Dumb, not-optimized RAID5 implementation is slower than writing to a single disk. This is generic RAID wisdom, nothing to do with any particular implementation. In the worst case scenario, the write operation on 6-disk RAID5 volume involves reading a data block from 5 drives, calculating XOR, and writing back one block of data and one block of checksum. Whichever way you do it, it ain't gonna be fast.
Still, he shouldn't be seeing less than 100MBps writes on the 3Ware Escalade 9500S series with its on-board ASIC and DRAM buffer.
I'm going to upgrade the firmware to the latest revision which should improve the write performance.
At least the reads are very accurate for his configuration. I'm curious how he is striping though? It might have been better to do a 12-disc RAID-5 and get close to 400MBps reads.
Or if performance was more important than efficiency, making one 6-disc volume RAID-10 would give close to 300MBps reads, 150MBps writes -- maybe higher.
I'm sticking with RAID-5 to maximize storage space while having some level of protection against drive failure.
Cheers! -Sean