Here is a sample of running iostat on a server that has a LUN from a SAN with multiple paths. I am specifying a device list that just grabs the bits related to the multi path device:

$ iostat -dxkt 1 2 sdf sdg sdh sdi dm-7 dm-8 dm-9 Linux 2.6.18-371.8.1.el5 (db21b.den.sans.org) 06/20/2014

Time: 02:30:23 PM Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util sdf 0.66 52.32 3.57 34.54 188.38 347.52 28.13 0.14 3.62 0.87 3.31 sdg 0.66 52.29 3.57 34.56 189.79 347.48 28.18 0.14 3.72 0.87 3.32 sdh 0.00 0.00 0.00 0.00 0.00 0.00 14.19 0.00 2.90 2.90 0.00 sdi 0.00 0.00 0.00 0.00 0.00 0.00 14.19 0.00 2.87 2.87 0.00 dm-7 0.00 0.00 8.46 173.75 378.17 695.00 11.78 3.41 18.68 0.35 6.46 dm-8 0.00 0.00 8.46 173.75 378.17 695.00 11.78 3.41 18.68 0.36 6.47 dm-9 0.00 0.00 8.46 173.75 378.17 695.00 11.78 3.41 18.68 0.36 6.48

Time: 02:30:24 PM Device: rrqm/s wrqm/s r/s w/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util sdf 0.00 54.00 7.00 48.00 88.00 408.00 18.04 0.12 2.11 1.20 6.60 sdg 0.00 13.00 1.00 26.00 4.00 156.00 11.85 0.01 0.52 0.48 1.30 sdh 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 sdi 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 dm-7 0.00 0.00 8.00 141.00 92.00 564.00 8.81 0.25 1.69 0.53 7.90 dm-8 0.00 0.00 8.00 141.00 92.00 564.00 8.81 0.25 1.70 0.54 8.00 dm-9 0.00 0.00 8.00 141.00 92.00 564.00 8.81 0.25 1.70 0.54 8.00

sdf,sdg,sdh,sdi - four paths for LUN (sdf and sdg are the active paths) dm-7 - device-mapper pseudo device for the mpath device dm-8 - device-mapper pseudo-device for the partition spanning the entire mpath device dm-9 - device-mapper pseudo-device for the LVM LV created on the mpath device

The first sample from iostat is the historical data so lets ignore it. The second sample are the stats for a 1 second interval.

I see the stats for sdf and sdg are roughly equal but they differ — they are the two active paths that are both being used.

I see the stats for dm-7, dm-8 and dm-9 are almost completely identical - makes sense as they really represent the same “disk”.

What confuses me is the fact that all the stats for sdf/sdg don’t add up to be equivalent to the dm-[789] devices

For the rkB/s and wkB/s columns, the numbers for sdf and sdg add up to equal the numbers for dm-9.

But for the first four columns:

Column sdf/sdg dm-9 ====== ============= ==== rrqm/s 0.0 + 0.0 = 0.0 0 wrqm/s 54.0 +13.0 = 67.0 0 Very different r/s 7.0 + 1.0 = 8.0 8.0 w/s 48.0 + 26.0 = 74.0 141.0 Very different

So read data matches and write data diverges

Which numbers should I go with? The physical devices or the logical device?

Thanks

David Goldsmith