[CentOS] ext3 filesystems larger than 8TB

[Ross S. W. Walker]

Monty Shinn wrote:
> 
> Ross,
> 
> We basically store video image sequences (edited and source) and 
> audio/video files on our servers.  We are an editing and broadcast 
> design facility, doing mostly HD work.  The files are relatively large, 
> and there are a lot of them.
> 
> I am trying to "max out" our current server population, moving from 250 
> and 500 gig drives to the Seagate 1TB enterprise (ES.2) SATA drives 
> using the 12 port 3ware raid card.
> 
> I have at least 4 servers that I am wanting to upgrade this way.
> 
> They're just file servers running NFS and Samba.
> 
> Do I *need* a 10TB partition?  No, not really.  I could segment into 2 
> 5TB partitions if needed, and I may still end up doing that.  I am 
> beginning to wonder if the >8TB ext3 limit has been vetted enough.  It 
> is just easier for the users if it was one partition.
> 
> I have to say when the mkfs.ext3 code hasn't been changed to allow >8TB 
> partitions without adding the -F, (which did seem to work) it gives me 
> pause.
> 
> Naively perhaps, I didn't think it would be an issue.

Makes sense, does NFS support sharing such large volumes? I suppose
that will depend on both the server version of NFS and the client,
but it's something you need to keep in mind.

I think for a large file file system xfs is probably what you want,
but you will want to run CentOS 64-bit with the 8k stacks to see
it's full robustness and stability.

Some people think xfs is good everywhere, but that's simply not
true, I always recommend putting the OS on ext3 and then choosing
the file system for your application data that best suits the
application. Basically you have ext3, jfs, xfs, gfs and ocfs, the
last 2 being clustered file systems. ext3 is good because it is
widely supported and performs well under mixed work load, jfs is
supposedly excellent if you have a lot of small files like a
mail/news server, xfs for large files and of course gfs or ocfs
for clusters that need simultaneous file system access from
multiple nodes (but they are slower due to locking overhead).

If you have volumes over 8TB then you really need to use either
jfs or xfs depending on the application and if you are using
xfs I highly recommend you run 64-bit for stability reasons.

-Ross

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.