On Mon, 2005-08-22 at 17:35 -0500, Sean Staats wrote:
We recently bought a 32-bit Xeon system with a 12-port
3Ware RAID card
and a dozen 500GB drives. We wanted to create 4TB drive arrays; however, we soon discovered that there is about a 2.2TB
drive array size
limit on 32-bit hardware. Does that sound correct?
Yes, 2^40 = 2TiB ~ 2.2TB (2.2 * 10^12). This is a PC geometry issue, although Linux can get around it.
Would replacing the 32-bit mobo/cpu with a 64-bit mobo/cpu
allow us to
use drive arrays larger than 2.2TB?
Actually it's a 3Ware question because 3Ware has an intelligent ASIC on- board. It's driving the disk array, not Linux. It's merely presenting the disk array as a block, and Linux talks to the ASIC, not the disks.
-- Bryan J. Smith b.j.smith@ieee.org http://thebs413.blogspot.com
The best things in life are NOT free - which is why life is easiest if you save all the bills until you can share them with the perfect woman
If you are running the latest 2.6 kernel you will not have this limitation. Also - make sure you have the latest 3ware firmware/driver.
/dev/sdb1 3.7T 2.2T 1.6T 59% /data
Linux storage1.******.com 2.6.11.12 #1 Mon Jun 20 10:40:15 PDT 2005 i686 i686 i386 GNU/Linux
Don't use ext2 or ext3 as your fs.
Take a look at http://www.3ware.com/kb/article.aspx?id=11920 as well.
--
On 8/22/05, Chris Mauritz chrism@imntv.com wrote:
Cletus Murphy wrote:
Don't use ext2 or ext3 as your fs.
Why not?
Cheers,
1. Because it will give you terrible gas.
2. I'm not going to open this up for a big FS thread but xfs/reiserfs performs much better on large partitions then ext2/3 in my humble experience - I also seem to remember that I could not create a partition larger then 4TB with ext3 on my system, though, I suffer from fetal alcohol syndrome so maybe I was doing something wrong.
hello. --
On Mon, 2005-08-22 at 16:16 -0700, Cletus Murphy wrote:
- I'm not going to open this up for a big FS thread but xfs/reiserfs
performs much better on large partitions then ext2/3 in my humble experience
Which ReiserFS? 3? 4? What about ReiserFS compatibility issues with various kernel interfaces? In those cases, Ext3 _is_ better because ReiserFS isn't an option.
Red Hat will not support ReiserFS until Hans starts supporting those interfaces. He won't, and compatibility with those interfaces are a "bread'n butter" for Red Hat, something that keeps me away from SuSE (and even SuSE admitted was a sore spot for their ReiserFS support back in 2000).
Now XFS on-the-other-hand, I think Red Hat really needs to wake up to. There are serious size/scalability limitations to Ext3 that XFS has solved very nicely for a long time. Red Hat really needs to start augmenting Ext3 support with XFS, and why they don't, I haven't heard one single, good answer.
XFS supports all the same kernel interfaces as Ext3, and has a better track record on many.
On 8/22/05, Bryan J. Smith b.j.smith@ieee.org wrote:
On Mon, 2005-08-22 at 16:16 -0700, Cletus Murphy wrote:
- I'm not going to open this up for a big FS thread but xfs/reiserfs
performs much better on large partitions then ext2/3 in my humble experience
Which ReiserFS? 3? 4? What about ReiserFS compatibility issues with various kernel interfaces? In those cases, Ext3 _is_ better because ReiserFS isn't an option.
Red Hat will not support ReiserFS until Hans starts supporting those interfaces. He won't, and compatibility with those interfaces are a "bread'n butter" for Red Hat, something that keeps me away from SuSE (and even SuSE admitted was a sore spot for their ReiserFS support back in 2000).
Now XFS on-the-other-hand, I think Red Hat really needs to wake up to. There are serious size/scalability limitations to Ext3 that XFS has solved very nicely for a long time. Red Hat really needs to start augmenting Ext3 support with XFS, and why they don't, I haven't heard one single, good answer.
XFS supports all the same kernel interfaces as Ext3, and has a better track record on many.
-- Bryan J. Smith b.j.smith@ieee.org http://thebs413.blogspot.com
The best things in life are NOT free - which is why life is easiest if you save all the bills until you can share them with the perfect woman
Initialy I tested with ReiserFS v3 - but at the end of the day I went with xfs.
--
Cletus Murphy wrote:
Don't use ext2 or ext3 as your fs.
Why not?
- Because it will give you terrible gas.
I see. And all this time, I thought it was the Armagnac....
- I'm not going to open this up for a big FS thread but xfs/reiserfs
performs much better on large partitions then ext2/3 in my humble experience - I also seem to remember that I could not create a partition larger then 4TB with ext3 on my system, though, I suffer from fetal alcohol syndrome so maybe I was doing something wrong.
Performs better for what? For my uses, mostly moving around uncompressed video, I don't seem to have any problems at all with ext3 using either software RAID or 3Ware RAID cards. As a matter of fact, I can state with a high degree of confidence that disk I/O hasn't been one of my bottlenecks to date. I'm mostly waiting on cpu cycles for encoding/rendering. I haven't needed to create a partition greater than 4TB yet. The biggest I've gone so far is a RAID0 software stripe of two 3.2gig hardware RAID0 partitions. These are for "scratch disks" for data that's relatively easy to reproduce so I don't bother with RAID 3/4/5. For data that's vitally important or hard to reproduce, I stash that on a RAID5 array (that gets backed up to a redundant RAID5 array). I suspect vanishingly few numbers of people have occasion to create partitions greater than 4tb. Let's see, that's 10+ 400gig drives. Don't get me wrong, I'd love to have petabytes of storage laying around, but the logistics of going over about 3TB per partition for most folks (at least with current hard drive sizes) are a bit out of reach. When this problem comes even remotely close to biting me in the hiney, THEN I'll worry about it. Until then, it's just a strawman.
Best regards,
On Mon, 2005-08-22 at 15:54 -0700, Cletus Murphy wrote:
If you are running the latest 2.6 kernel you will not have this limitation. Also - make sure you have the latest 3ware firmware/driver. /dev/sdb1 3.7T 2.2T 1.6T 59% /data Linux storage1.******.com 2.6.11.12 #1 Mon Jun 20 10:40:15 PDT 2005 i686 i686 i386 GNU/Linux Don't use ext2 or ext3 as your fs. Take a look at http://www.3ware.com/kb/article.aspx?id=11920 as well.
Excellent. I've only been running with 2.2TB or less volumes on my 3Ware cards to date.
-
Bryan J. Smith -
Chris Mauritz -
Cletus Murphy