Hi Nicolas,
While this doesn't exactly answer your question, I was wondering what scheduler you were using on your GFS2 (Note: I have not used this file system before) block. You can find this by issuing 'cat /sys/block/<insert block device>/queue/scheduler' ?
By default the system uses cfq, which will show up as [cfq] when catting the scheduler as I showed above. This is not the most optimal scheduler for a webserver. In most cases you'd be better off with deadline or noop. Not being familiar with GFS2 myself, I did skim this article, which makes me think noop would be the better choice:
http://www.redhat.com/archives/linux-cluster/2010-June/msg00027.html
This could be why you are seeing the processes waiting on I/O.
Chad M. Gross
On Tue, Sep 20, 2011 at 2:55 PM, Nicolas Ross rossnick-lists@cybercat.cawrote:
Hi list !
We have a very busy webserver hosted in a clustered environment where the document root and data is on a GFS2 partition off a fiber-attached disk array.
Now on busy moments, I can see in htop, nmon that there is a fair percentage of cpu that is waiting for I/O. In nmon, I can spot that the most busy block device correspond to our gfs2 partition where many times, it shows that it's 100% busy and is read all along.
Now, I want to know what files are being waited for. With lsof I can get a listing of open files, but it doesn't gives me if a file is just opended in ram or if it's being waited for...
What tools besides lsof, nmon, htop, atop can help me find that info ?
I am under RHEL/CentOS 6.1.
Thanks
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos