Chan Chung Hang Christopher wrote:
Benjamin Franz wrote:
Yup. 8 way RAID1 for the OS, 8 way RAID6 for the data. I was hoping when I setup the 8-way RAID1 for the OS that I would get really good read speeds since md is supposed to stripe reads from RAID1, but in practice the RAID6 completely kills it for read performance (~61 MB/sec from the RAID1 partition vs ~200 MB/sec from the RAID6 partition).
What are you running? I think there was a patch that evened out the reads across all members as it would at times solely read from one and then another...
I'm running fully up-to-date CentOS 5.4, kernel 2.6.18-164.15.1. The test was done with a default run of bonnie++. Watching the disk I/O while it ran suggested that it was only using some of the disks at a time to read (but changing which ones every few seconds).
8 x RAID1
Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP pbox16.freerun. 16G 586 95 17263 5 13016 1 2263 97 60793 3 644.2 4 Latency 203ms 188s 1521ms 14714us 557ms 1265ms Version 1.96 ------Sequential Create------ --------Random Create-------- pbox16 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 12418 16 +++++ +++ 10999 12 27730 34 +++++ +++ 29916 32 Latency 109us 638us 681us 200us 25us 39us
8 x RAID6 Version 1.96 ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP pbox16.freerun. 16G 611 96 90060 15 55711 14 2407 96 200901 27 522.3 15 Latency 13157us 604ms 1727ms 32420us 142ms 73131us Version 1.96 ------Sequential Create------ --------Random Create-------- pbox16 -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 10298 16 +++++ +++ 24208 32 30395 45 +++++ +++ 32511 43 Latency 18504us 598us 613us 87us 25us 121us 1.96,1.96,pbox16,1,1269604576,16G,,611,96,90060,15,55711,14,2407,96,200901,27,522.3,15,16,,,,,10298,16,+++++,+++,24208,32,30395,45,+++++,+++,32511,43,13157us,604ms,1727ms,32420us,142ms,73131us,18504us,598us,613us,87us,25us,121us