We have an appliance that uses the CentOS 4.4 Plus kernel with XFS on an EMC RAID array that performs about ¼ or less as well with cached i/o than with direct i/o. With direct i/o we get about 32Mb/s throughput or better, but if we enable the cache for the EMC RAID, we only get 8Mb/s or less. This doesn't happen if we use JFS, where we get a little less than 32Mb/s throughput with or without cache enabled, on the same hardware and CentOS, but there are some features in XFS that we use that are not available in JFS..
Any thoughts on this?
Thanks.
On Thu, 8 Mar 2007 at 4:50pm, Mark Hull-Richter wrote
We have an appliance that uses the CentOS 4.4 Plus kernel with XFS on an
The first thing you want to ensure is that you're using the XFS kernel module avaiable in the plus repository and not XFS as compiled in the plus kernel. The module code is *much* newer and less likely to be broken.
Good point (use a good XFS), but it seems I misspoke myself:
Whether we use JFS or XFS, when the cache is enabled, we get less than 50% of the performance we see when we use direct i/o. Direct i/o is not an option for general use, so we need to understand why cached i/o is so slow.
We write huge files sequentially on the whole and as a rule in our app, so that is probably a factor. We believe that our i/o overwhelms the block level elevator algorithm, and that the cache flush algorithm isn't particularly concerned with sequential i/o.
Can anyone shed some light on this?
Thanks.
We are using the deadline method - cached i/o still runs 1/2 the speed of direct. Also, for current testing, we've been using JFS....
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Jim Perrin Sent: Friday, March 09, 2007 1:32 PM To: CentOS mailing list Subject: Re: [CentOS] Performance problem with CentOS and XFS on EMC
Sure. Don't use XFS.
Or use a different I/O method, like deadline.
Please don't top post.
On 3/9/07, Mark Hull-Richter mhull-richter@datallegro.com wrote:
We are using the deadline method - cached i/o still runs 1/2 the speed of direct. Also, for current testing, we've been using JFS....
Okay... so you're using 2 different filesystems known to be unsupported in the upstream OS, and you're wondering why you're getting bad performance?
While XFS has been backported (great thanks to those responsible and not to belittle their efforts) and is functional, it's not been heavily debugged or tested to any great extent beyond 'yep it works and did not eat my data'.
To futher the issue, vendors supporting RH don't test those alternate filesystems against RHEL because they're not supported in the distro.
You appear to be using a hammer to pound screws. You may really want to take a moment to sit back and question the sanity involved in what you're doing.
One other nit-pick as I'm a little frustrated at this point. PLEASE PROVIDE ALL RELEVANT DATA to those trying to assist you. If you've used other elevators, STATE THAT so that we don't waste our effort telling you something you've already tried.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Jim Perrin Sent: Friday, March 09, 2007 2:55 PM To: CentOS mailing list Subject: Re: [CentOS] Performance problem with CentOS and XFS on EMC
Please don't top post.
Okay. Personally I prefer to get the meat at the top, but I supposed context could be relevant once in a while....
Okay... so you're using 2 different filesystems known to be unsupported in the upstream OS, and you're wondering why you're getting bad performance?
No, I'm wondering why I'm getting such a disparity in performance between direct i/o and cached i/o using two different files systems (same issue on both file systems, different issue than 'why is this file system not performing well/better?').
You appear to be using a hammer to pound screws. You may really want to take a moment to sit back and question the sanity involved in what you're doing.
Ever heard of drywall screws? :-)
One other nit-pick as I'm a little frustrated at this point. PLEASE PROVIDE ALL RELEVANT DATA to those trying to assist you. If you've used other elevators, STATE THAT so that we don't waste our effort telling you something you've already tried.
Apologies - one detail that I overlooked until you reminded me. I'm still relatively new to this project, RHEL/CentOS and so on. I may have mentioned this once or twice before....
-
Jim Perrin -
Joshua Baker-LePain -
Mark Hull-Richter