[CentOS] Very slow disk I/O

Thu Jan 29 13:07:44 UTC 2015
Jatin Davey <jashokda at cisco.com>

On 1/28/2015 11:07 PM, John R Pierce wrote:
> On 1/28/2015 4:32 AM, Jatin Davey wrote:
>> I am using RHEL 6.5 on my server.
>> From top command i can see that the processors in my server are 
>> spending a lot of time on wait for I/O.
>> I can see high percentage in terms of 30-50% on "wa" time.
>> Here is the df output about the disk space in my system:
>> **********
>> [root at localhost images]# df
>> Filesystem      1K-blocks      Used  Available Use% Mounted on
>> /dev/sda1      1915071844 103227908 1714563936   6% /
>> tmpfs            32931472         0   32931472   0% /dev/shm
>> **********
>> Could someone point me on how to improve the disk I/O on my server 
>> and reduce the wait time on I/O. 
> it appears you have everything on a single file system on a single 2TB 
> drive?
Yes , entire disk is mounted on a single file system. Will this cause 
high wait time for I/O operations ? Do you think splitting it into 
multiple partitions would help ? Appreciate if you can enlighten me in 
this regard or provide pointers to it.
> I'm going to guess thats a desktop grade SATA disk spinning at most at 
> 7200rpm (many are slower, 'green' disks are often 5400rpm).
Yes , it is a  SATA disk. I am not sure of the speed. Can you tell me 
how to find out this information ? Additionally we are using RAID 10 
configuration with 4 disks.
> what sort of workload are you running?
Our workload is a bit high on disk reads / writes and hence i am seeing 
slowness in it when using the top command. We basically run VMs on the 
RHEL 6.5 host. Our target is to house 5-6 application VMs on this host 
and ship this as a product on a rack server.
> our high disk IO servers, doing things like transactional databases, 
> have many 10000rpm or 15000 rpm drives on a SAS2 raid card with 1GB or 
> 2GB of write-back-cache (battery or flash backed).
Hi John

Please see my responses inline. Appreciate your inputs in this regard.