[CentOS] mkfs.ext3 on a 9TB volume

Matt Hyclak hyclak at math.ohiou.edu
Wed Sep 14 14:29:33 UTC 2005


On Thu, Sep 15, 2005 at 12:28:01AM +1000, Nick Bryant enlightened us:
> > > On 9/12/05, Joshua Baker-LePain <jlb17 at duke.edu> wrote:
> > > > As I mentioned, I'm running centos-4, which, as we all know, is based
> > off
> > > > RHEL 4.  If you go to <http://www.redhat.com/software/rhel/features/>,
> > > > they explicitly state that they support ext3 FSs up to 8TB.
> > >
> > > Wow! Odd! RH says 8TB but ext3 FAQ says 4TB.
> > 
> > I wouldn't call it that odd.  RH patches their kernels to a fair extent,
> > both for stability and features.
> > 
> > > >From my personal testing on CentOS 4.1, you can't go over 4TB without
> > kludging.
> > >
> > > > I then did a software RAIDO across them, and finally:
> > > >
> > > > mke2fs -b 4096 -j -m 0 -R stride=1024 -T largefile4 /dev/md0
> > >
> > > Joshua, thanks for the reply on this.
> > > There's something kludgy about having to do softraid across 2
> > > partitions before formatting. It adds a layer of complexity and
> > > reduces reliability. Is that the trick RH recommended to go up to 8TB?
> > 
> > Err, it's not a kludge and it's not a trick.  Those 2 "disks" are hardware
> > RAID5 arrays from 2 12 port 3ware 9500 cards.  I like 3ware's hardware
> > RAID, and those are the biggest (in terms of ports) cards 3ware makes.
> > So, I hook 12 disks up to each card, and the OS sees those as 2 SCSI
> > disks.  I then do the software RAID to get 1) speed and 2) one partition
> > to present to the users.  Folks (myself included) have been doing this for
> > years.
> > 
> > The one gotcha in this setup (other than not being able to boot from the
> > big RAID5 arrays, since each is >2TiB) is that the version of mdadm
> > shipped with RHEL4 does not support array members bigger than 2TiB.  I had
> > to upgrade to an upstream release to get that support.
> 
> Just out of interest, and to complicate the matter even more, does anyone
> know what the upper limit of GFS is?
> 

From what I've been reading, there's an 8TB limit of all GFS file systems in
a cluster.

http://www.redhat.com/docs/manuals/csgfs/browse/rh-gfs-en/s1-sysreq-fibredevice.html

Matt

-- 
Matt Hyclak
Department of Mathematics 
Department of Social Work
Ohio University
(740) 593-1263
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: <http://lists.centos.org/pipermail/centos/attachments/20050914/01b369ae/attachment.sig>


More information about the CentOS mailing list