Thanks for the reply, Joshua. Looks like you're right about 3Ware recommending that NCQ be disabled. The settings listed at http://www.3ware.com/KB/article.aspx?id=14952 show NCQ disabled. More importantly, it shows that I *should* be able to get 185MB/Sec writes with four disks. http://www.3ware.com/kb/article.aspx?id=11050 also mentions making the blockdev adjustment. First off, I just tried XFS without changing anything else. I downloaded the XFS kernel modules and the XFS tools from centosplus, created a 100GB logical volume and formatted it with xfs. The results: ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP Beryl 10G:64k 56595 91 132091 23 42049 6 45990 72 153534 13 195.7 1 Beryl 10G:64k 60359 94 127813 24 42298 6 48124 73 162938 14 191.0 1 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP Beryl 16 1990 9 +++++ +++ 3019 14 1880 9 +++++ +++ 643 3 Beryl 16 1802 8 +++++ +++ 2776 12 1696 8 +++++ +++ 676 3 Whoa! Block writes jumped to 130MB/Sec with no other changes. XFS Rocks! Next, I disabled NCQ and set set StorSave to "Balance": ------Sequential Output------ --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP Beryl 10G:64k 58142 93 202483 35 55925 8 46244 73 160890 14 200.6 2 Beryl 10G:64k 60640 94 204090 38 55043 8 48603 74 168764 15 200.8 1 ------Sequential Create------ --------Random Create-------- -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files:max:min /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP Beryl 16 3112 15 +++++ +++ 4276 19 2673 13 +++++ +++ 811 4 Beryl 16 3242 16 +++++ +++ 4249 19 2903 15 +++++ +++ 828 4 Double Whoa! Block writes increased to 202MB/Sec. Very acceptable. I'll play with blockdev next and see if I can increase the read speeds. Thanks again for your suggestions. Kirk Bocek Joshua Baker-LePain wrote: > On Sat, 30 Sep 2006 at 8:56pm, Kirk Bocek wrote > >> 3Ware's site advertises 300MB/Sec writes using 8 disks on the PCI >> Express version of this card (the 9580 I think.) I'm guessing 4x PCI >> Express has more bandwidth but 133Mhz 64-bit PCI-X is no slouch. The >> only two things I need to change are to put the disks in 3GB/Sec SATA >> mode (they're in 1.5GB/Sec mode right now) and to enable memory >> interleaving on the motherboard (I only have two sticks of memory, >> this M/B needs 4 for interleaving.) But I don't think these things >> should drop me down to the write performance of a single IDE disk. >> >> Does anyone have any suggestions on improving write performance? > > 1) Make sure the firmware on the board is up to date. > > 2) Turn off NCQ (last I knew, this was 3ware's advice). > > 3) Try putting the ext3 journal on another device. > > 4) Use XFS. > > The last one is not just a prejudice. There really seems to be some > sort of odd (and rather bad) interaction between 3ware RAID5 and ext3. > Going to XFS really helps (especially with writes). > > Oh, and be sure to 'blockdev --setra 16384' that device, which will help > with reads. 16384 is what works for me -- YMMV, of course. >