[CentOS] iostat results for multi path disks

Fri Jun 20 14:47:01 UTC 2014
David Goldsmith <dgoldsmith at sans.org>

Here is a sample of running iostat on a server that has a LUN from a SAN with multiple paths.  I am specifying a device list that just grabs the bits related to the multi path device:

$ iostat -dxkt 1 2 sdf sdg sdh sdi dm-7 dm-8 dm-9
Linux 2.6.18-371.8.1.el5 (db21b.den.sans.org) 	06/20/2014

Time: 02:30:23 PM
Device:         rrqm/s   wrqm/s   r/s   w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sdf               0.66    52.32  3.57 34.54   188.38   347.52    28.13     0.14    3.62   0.87   3.31
sdg               0.66    52.29  3.57 34.56   189.79   347.48    28.18     0.14    3.72   0.87   3.32
sdh               0.00     0.00  0.00  0.00     0.00     0.00    14.19     0.00    2.90   2.90   0.00
sdi               0.00     0.00  0.00  0.00     0.00     0.00    14.19     0.00    2.87   2.87   0.00
dm-7              0.00     0.00  8.46 173.75   378.17   695.00    11.78     3.41   18.68   0.35   6.46
dm-8              0.00     0.00  8.46 173.75   378.17   695.00    11.78     3.41   18.68   0.36   6.47
dm-9              0.00     0.00  8.46 173.75   378.17   695.00    11.78     3.41   18.68   0.36   6.48

Time: 02:30:24 PM
Device:         rrqm/s   wrqm/s   r/s   w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sdf               0.00    54.00  7.00 48.00    88.00   408.00    18.04     0.12    2.11   1.20   6.60
sdg               0.00    13.00  1.00 26.00     4.00   156.00    11.85     0.01    0.52   0.48   1.30
sdh               0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sdi               0.00     0.00  0.00  0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
dm-7              0.00     0.00  8.00 141.00    92.00   564.00     8.81     0.25    1.69   0.53   7.90
dm-8              0.00     0.00  8.00 141.00    92.00   564.00     8.81     0.25    1.70   0.54   8.00
dm-9              0.00     0.00  8.00 141.00    92.00   564.00     8.81     0.25    1.70   0.54   8.00


sdf,sdg,sdh,sdi - four paths for LUN (sdf and sdg are the active paths)
dm-7 - device-mapper pseudo device for the mpath device
dm-8 - device-mapper pseudo-device for the partition spanning the entire mpath device
dm-9 - device-mapper pseudo-device for the LVM LV created on the mpath device

The first sample from iostat is the historical data so lets ignore it.  The second sample are the stats for a 1 second interval.

I see the stats for sdf and sdg are roughly equal but they differ — they are the two active paths that are both being used.

I see the stats for dm-7, dm-8 and dm-9 are almost completely identical - makes sense as they really represent the same “disk”.

What confuses me is the fact that all the stats for sdf/sdg don’t add up to be equivalent to the dm-[789] devices

For the rkB/s and wkB/s columns, the numbers for sdf and sdg add up to equal the numbers for dm-9.

But for the first four columns:

Column	sdf/sdg			dm-9
======	=============	====
rrqm/s	0.0 + 0.0 = 0.0		0		
wrqm/s	54.0 +13.0 = 67.0	0		Very different
r/s		7.0 + 1.0 = 8.0		8.0
w/s		48.0 + 26.0 = 74.0	141.0	Very different

So read data matches and write data diverges


Which numbers should I go with?  The physical devices or the logical device?

Thanks

David Goldsmith
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 204 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://lists.centos.org/pipermail/centos/attachments/20140620/592906ce/attachment-0004.sig>