[CentOS] Very slow disk I/O

[Gordon Messmer]

On 01/29/2015 07:21 PM, Jatin Davey wrote:
>
> [root at localhost ~]# lspci | grep RAID
> 05:00.0 RAID bus controller: LSI Logic / Symbios Logic MegaRAID SAS-3
> 3108 [Invader] (rev 02)

I'd expect some pretty good numbers from that with a 4 disk RAID 10.  As 
before, SSD will perform better.  If you need 2TB of space at similar 
cost, maybe consider 8 500GB drives instead of 4 1TB drives.

Since your workload is in VMs, you'll also see a large (potentially very 
large) performance benefit from a change to your filesystem layout.  I 
recommend something closer to the default: one partition for /boot, and 
a second partition spanning the rest of the disk (RAID volume).  The 
second partition should be part of an LVM group, your root filesystem 
should be limited to what it needs.  Maybe a 40GB root FS.  Use LVM 
backed VMs instead of file backed VMs.

When you use file backed VMs, all of your disk IO goes through the 
filesystem layer, and possibly through the journal, which amplifies your 
write count.  That's costly.

Beyond that, you might just want to test the system and make sure that 
it's performing as expected.  Figure out how much RAM is on your RAID 
controller and how much is available to the host system.  Shut down all 
of your VMs and run "bonnie++" with a data size at least 2x the size of 
system RAM + RAID controller RAM.