[CentOS] ext3 filesystems larger than 8TB

[Monty Shinn]

Ross S. W. Walker wrote:
> Ray Van Dolson wrote:
>> On Fri, May 02, 2008 at 02:36:41PM -0500, Monty Shinn wrote:
>>> Greetings.
>>>
>>> I am trying to create a 10TB (approx) ext3 filesystem.  I am able to 
>>> successfully create the partition using parted, but when I try to use 
>>> mkfs.ext3, I get an error stating there is an 8TB limit for ext3 
>>> filesystems.
>>>
>>> I looked at the specs for 5 on the "upstream" vendor's website, and they  
>>> indicate that there is a 16TB limit on ext3.
>>>
>>> Has anyone been able to create a ext3 filesystem larger than 8TB?
>>>
>>> If ext3 isn't an option, has anyone used the kmod-xfs-smp.i686 module 
>>> mentioned on the centos site?  Surely it doesn't have an 8TB limit...
>>>
>>> specs:
>>>
>>> Centos 5.1
>>> kernel-2.6.18-53.1.14.el5
>>> 3Ware 9550SXU 12port raid card
>>>
>>> Am I just walking into a big nightmare?
>>>
>>> Any thoughts would be greatly appreciated.
>>>
>> I think it *theoretically* should work as ext3 should to 16TB.
>> However, we ran into issues with userland tools and such.  Possibly
>> related to x86_64 vs i386 stuff, but in the end to avoid continued
>> troubleshooting we just used centosplus + jfs.  Works 
>> perfectly for our
>> 10TB filesystem.
> 
> I'm curious what you store that you need 10TB of linear storage?
> 
> I have had 4-6-8TB storage systems, but the storage was always
> divvied up between different applications.
> 
> -Ross
> 
> ______________________________________________________________________
> This e-mail, and any attachments thereto, is intended only for use by
> the addressee(s) named herein and may contain legally privileged
> and/or confidential information. If you are not the intended recipient
> of this e-mail, you are hereby notified that any dissemination,
> distribution or copying of this e-mail, and any attachments thereto,
> is strictly prohibited. If you have received this e-mail in error,
> please immediately notify the sender and permanently delete the
> original and any copy or printout thereof.
> 
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos

Ross,

We basically store video image sequences (edited and source) and 
audio/video files on our servers.  We are an editing and broadcast 
design facility, doing mostly HD work.  The files are relatively large, 
and there are a lot of them.

I am trying to "max out" our current server population, moving from 250 
and 500 gig drives to the Seagate 1TB enterprise (ES.2) SATA drives 
using the 12 port 3ware raid card.

I have at least 4 servers that I am wanting to upgrade this way.

They're just file servers running NFS and Samba.

Do I *need* a 10TB partition?  No, not really.  I could segment into 2 
5TB partitions if needed, and I may still end up doing that.  I am 
beginning to wonder if the >8TB ext3 limit has been vetted enough.  It 
is just easier for the users if it was one partition.

I have to say when the mkfs.ext3 code hasn't been changed to allow >8TB 
partitions without adding the -F, (which did seem to work) it gives me 
pause.

Naively perhaps, I didn't think it would be an issue.

Thanks,

Monty