On Thu, 20 Oct 2005 at 10:27am, Aleksandar Milivojevic wrote
Quoting Joshua Baker-LePain jlb17@duke.edu:
Wonderful. I'm testing RAID5 now, for which I had high hopes.
Well, my guess is that there'll be fix for this by the time next kernel version is out. Obviously it is an bug that should be fixed.
There weren't any RAID6 specific bugs I could find in bugzilla. I'll probably file one once I test on a 2nd system and with the most recent kernel (all my testing has been with 2.6.9-11 as our very excellent campus-based distro hasn't pushed 4.2 yet).
BTW, any particular reason not to use hardware RAID on 3ware?
That was the initial plan. But I can't get anything even resembling decent speeds out of them. These systems have 2 7500-8 boards in 'em (on separate PCI buses). I had been running RH7.3 (!!) and XFS with the boards in hardware RAID5 and a software RAID0 stripe across those, and would get
100MB/s writes and >300MB/s reads. With centos-4 and ext3, I was getting
~30MB/s writes and ~200MB/s reads. The reads I'd be OK with, but the write speed is absurd. I tweaked all I could think of, but nothing helped much.
Yeah, mixing RAIDs like that sometimes can get you into trouble.
It's worked well in the past (obviously), and on my dual 9500 based system. It's the same general setup, and gets >150MB/s writes and nearly 400MB/s reads with centos-4 and ext3.
I guess 3ware was not rebuilding anything while you were testing? Have you tested the speed of ext3 under RH7.3 (to rule out file system differences influencing your particular test)?
No, the 3wares weren't rebuilding. I can't seem to find any RH7.3 ext3 benchmarking I did. I *did* try the centosplus kernel and XFS with the mixed hardware/software RAID setup. It got ~100MB/s writes but the reads went down to ~150MB/s. I just don't trust the stack to handle XFS and NFS on a production server.
Is the speed OK if you use hardware RAID-5 on one card on its own (not mixing it with sofware RAID-0 like you had under RH-7.3)? If it is, try this configuration instead. Create two hardware RAID-5 volumes (one on each card). Then use fdisk to mark them as Linux LVM (8e). Use pvcreate to create LVM physical volumes on them, put them into volume group (for example create a new one) and make a logical volume on them. Example:
I haven't tested a single board yet -- it'll be interesting to see what that yields. I hadn't thought of using LVM -- I may give that a shot. Thanks.