First of all, thanks to everybody for their responses on this thread. On Tue, 2005-11-01 at 10:31, centos-bounces at centos.org wrote: > Aleksandar Milivojevic <alex at milivojevic.org> wrote: > > Correct me if I'm wrong, but you got 233MB/s for reads (the > > block read test). > > Oh, good catch! I didn't even see that when responding (I > assumed he could interpret the bonnie benchmark). And if I > see that correctly, that was with a 12GiB file (on a system > that had 6GiB RAM). I was fixated on the per char read rate and didn't pay much attention to the block read rate. ;-) > > Assuming your disks can do 50MB/s sustained transfer rate > > each, you are preatty darn close to the theoretical maximum > > of (6 - 1) * 50MB/s = 250MB/s for 6 disk RAID5. At least the read speeds are performing as can be reasonably expected for this particular configuration. I am certainly happy with that result. > On reads, yes. 3Ware is clearly leveraging the ASIC's > non-blocking I/O for reads from RAID-5, which basically act > like RAID-0. > > > RAID5 as such is bad choice for file systems that will have > > more than about 30% of writes (out of total I/O). > > He still should be seeing at least 100MBps for RAID-5 writes > on a 3Ware Escalade 9500S with 6-discs (180MBps is about the > maximum for RAID-5 writes on the 9500S' ASIC with DRAM). The > ASIC is fairly good at sequential writes to RAID-5, and there > is enough DRAM to buffer all but the heaviest of random I/O. > > Still, the new 9550SX series has a PowerPC. AMCC's influence > is clearly being pressed on their 3Ware acquisition, as they > are _the_ company for the IBM embedded PowerPC 400 line now. > The 9550SX is supposed to be cable of 380MBps for RAID-5 > writes -- double the 9500S best benchmarks. > > > If most of the I/O will be writes, and you care about > > performance, you should use RAID-10. > > Yep, mega-dittos on that point. > > > Remember, writes to Dumb, not-optimized RAID5 > > implementation is slower than writing to a single > > disk. This is generic RAID wisdom, nothing to do with > > any particular implementation. In the worst case scenario, > > the write operation on 6-disk RAID5 volume involves > > reading a data block from 5 drives, calculating XOR, > > and writing back one block of data and one block of > > checksum. Whichever way you do it, it ain't gonna be > > fast. > > Still, he shouldn't be seeing less than 100MBps writes on the > 3Ware Escalade 9500S series with its on-board ASIC and DRAM > buffer. I'm going to upgrade the firmware to the latest revision which should improve the write performance. > At least the reads are very accurate for his configuration. > I'm curious how he is striping though? It might have been > better to do a 12-disc RAID-5 and get close to 400MBps reads. > > Or if performance was more important than efficiency, making > one 6-disc volume RAID-10 would give close to 300MBps reads, > 150MBps writes -- maybe higher. I'm sticking with RAID-5 to maximize storage space while having some level of protection against drive failure. Cheers! -Sean