[CentOS] Setting up large (12.5 TB) filesystem howto?

Fri Aug 28 15:36:04 UTC 2009
Joshua Baker-LePain <jlb17 at duke.edu>

On Fri, 28 Aug 2009 at 1:03pm, Götz Reinicke - IT-Koordinator wrote

> fdisk and parted fail to create any information on the device or fail
> completely.

You can't use fdisk on a volume that large.  parted should work fine. 
What was the error you were getting (exactly)?  For a volume that large, 
you must use a GPT disk label, not the default msdos one.

> But, I can't create a filesystem on it:
>
> mkfs.ext3 -m 2 -j -O dir_index -v -b 4096 -L iscsi2lvol0
> /dev/mapper/VolGroup02-lvol0
>
>
> mke2fs 1.39 (29-May-2006)
> mkfs.ext3: Filesystem too large.  No more than 2**31-1 blocks
> 	 (8TB using a blocksize of 4k) are currently supported.

As has been pointed out, you need to use "-F" to force mkfs.ext3 to make a 
filesystem bigger than 8TB.  IMHO, this is misleading.  Filesystems up to 
16TB are fully supported in centos >5.1, so I don't see why the upstream 
vendor left the requirement for "-F" in mkfs.ext3.

> So my question: What is my missunderstanding or what's wrong with my
> system? Where are the real limits? Do I have to switch the OS to 64 Bit?

You do not have to switch to 64bit, and your setup should be fully 
supported.  Other folks have mentioned XFS, and that's an option.  But if 
you want to stay fully compatible with upstream, then ext3 is your only 
option.

-- 
Joshua Baker-LePain
QB3 Shared Cluster Sysadmin
UCSF