John Doe wrote:
Hi,
We have a storage server (HP DL360G5 + MSA20 (12 disks in RAID 6) on a SmartArray6400). 10 directories are exported through nfs to 10 clients (rsize=32768,wsize=32768,soft,intr,nosuid,proto=udp,vers=3). The server is apparently not doing much but... we have very high waiting IOs.
dstat show very little activity, but high 'wai'...
as others have said, iostat -x N for an N like 5 (5 seconds). ignore the first sample as its the average since boot, instead, look at the ongoing 5 second interval samples, and look for high await, svctm, and %util, as well as the rrqm/s and wrqm/s numbers, rather than sec/s, as sequential access is likely the least of your problems. raid6 does pretty poorly on heavy random write workloads
Also, IBM has a neat freeware system analysis tool called NMON (originally for AIX, ported to Linux) http://www.ibm.com/developerworks/aix/library/au-analyze_aix/ works sorta like a souped up 'top' but has per file system IO stats and stuff too. it can also accumulate stats over a long period into a CSV file, and they have an excel spreadsheet that loads said CSV file and cranks out a lot of fairly useful graphs. dunno if the excel spreadsheet works in OOcalc or not.