[CentOS-virt] disk i/o benchmarking inside vm

Thu Oct 20 06:13:29 UTC 2011
Charles Polisher <cpolish at surewest.net>

Mon, Oct 10, 2011, Dennis Jacobfeuerborn wrote:
<snip />
> The physical numbers are consistent and what I would expect to see from the 
> sata drives.
> The guests are minimal centos 6 installations so after booting they have 
> virtually no processes running that could influence the benchmarks in any 
> significant way. The hosts system is installed on /dev/sda so /dev/sd(b|c) 
> are not influenced by the host system either.
> The entire setup is arranged to make benchmarking mostly reliable.
> If there were minor temporary fluctuations I would blame some external 
> process but differences of almost 100% that are consistent for the lifetime 
> of the virtual machine do not fit such a scenario.

Hi Dennis,

Did you figure this out?

I'd suggest running something like smartctl -t long to rule out
underlying issues with a drive.

My favorite tool for testing the data path was Robin Miller's
dt, which now appears to be absent from the web.  I'll get you a
source tarball if you contact me off list.

Moving up the data chain to the kernel level, I wonder what
iostat would show while the benchmarks are running. Something
like -

    /path/to//startbenchmark &
    for i in `seq 1 10`; do
        sleep 30
        iostat -x >> /path/to/iostat.log
    done

While that's running, in another term, /usr/bin/top might 
show unexpected values for iowait/w_chan/process-status.

Perhaps the beginning block offset for the .img files are not
identical. If the alignment was unfortunate, perhaps the OS
needs to do an extra seek for every track it reads or some such.

Best regards,
-- 
Charles Polisher