[CentOS] RAID 5 setup?

Sat Mar 27 13:24:32 UTC 2010
Benjamin Franz <jfranz at freerun.com>

Chan Chung Hang Christopher wrote:
> Benjamin Franz wrote:
>   
>> Yup. 8 way RAID1 for the OS, 8 way RAID6 for the data. I was hoping when 
>> I setup the 8-way RAID1 for the OS that I would get really good read 
>> speeds since md is supposed to stripe reads from RAID1, but in practice 
>> the RAID6 completely kills it for read performance (~61 MB/sec from the 
>> RAID1 partition vs ~200 MB/sec from the RAID6 partition).
>>     
>
> What are you running? I think there was a patch that evened out the 
> reads across all members as it would at times solely read from one and 
> then another...
>
>   

I'm running fully up-to-date CentOS 5.4, kernel 2.6.18-164.15.1. The 
test was done with a default run of bonnie++. Watching the disk I/O 
while it ran suggested that it was only using some of the disks at a 
time to read (but changing which ones every few seconds).

8 x RAID1

Version  1.96       ------Sequential Output------ --Sequential Input- 
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  
/sec %CP
pbox16.freerun. 16G   586  95 17263   5 13016   1  2263  97 60793   3 
644.2   4
Latency               203ms      188s    1521ms   14714us     557ms    
1265ms
Version  1.96       ------Sequential Create------ --------Random 
Create--------
pbox16  -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
                 16 12418  16 +++++ +++ 10999  12 27730  34 +++++ +++ 
29916  32
Latency               109us     638us     681us     200us      25us      
39us

8 x RAID6
Version  1.96       ------Sequential Output------ --Sequential Input- 
--Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- 
--Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  
/sec %CP
pbox16.freerun. 16G   611  96 90060  15 55711  14  2407  96 200901  27 
522.3  15
Latency             13157us     604ms    1727ms   32420us     142ms   
73131us
Version  1.96       ------Sequential Create------ --------Random 
Create--------
pbox16  -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  
/sec %CP
                 16 10298  16 +++++ +++ 24208  32 30395  45 +++++ +++ 
32511  43
Latency             18504us     598us     613us      87us      25us     
121us
1.96,1.96,pbox16,1,1269604576,16G,,611,96,90060,15,55711,14,2407,96,200901,27,522.3,15,16,,,,,10298,16,+++++,+++,24208,32,30395,45,+++++,+++,32511,43,13157us,604ms,1727ms,32420us,142ms,73131us,18504us,598us,613us,87us,25us,121us

-- 
Benjamin Franz