We recently bought a 32-bit Xeon system with a 12-port 3Ware RAID card and a dozen 500GB drives. We wanted to create 4TB drive arrays; however, we soon discovered that there is about a 2.2TB drive array size limit on 32-bit hardware. Does that sound correct?
Would replacing the 32-bit mobo/cpu with a 64-bit mobo/cpu allow us to use drive arrays larger than 2.2TB?
Thanks.
On Mon, 2005-08-22 at 17:35 -0500, Sean Staats wrote:
We recently bought a 32-bit Xeon system with a 12-port 3Ware RAID card and a dozen 500GB drives. We wanted to create 4TB drive arrays; however, we soon discovered that there is about a 2.2TB drive array size limit on 32-bit hardware. Does that sound correct?
Yes, 2^40 = 2TiB ~ 2.2TB (2.2 * 10^12). This is a PC geometry issue, although Linux can get around it.
Would replacing the 32-bit mobo/cpu with a 64-bit mobo/cpu allow us to use drive arrays larger than 2.2TB?
Actually it's a 3Ware question because 3Ware has an intelligent ASIC on- board. It's driving the disk array, not Linux. It's merely presenting the disk array as a block, and Linux talks to the ASIC, not the disks.
Thank you for your response, Bryan. It appears that the 3Ware card is reporting the disk array size correctly. I've now created 2 arrays on the controller - one is 1999.9 GB (/dev/sda) and the other is 2499.9 GB (/dev/sdb) as reported by 'fdisk -l':
[root@anchor ~]# fdisk -l /dev/sda
Disk /dev/sda: 1999.9 GB, 1999957393408 bytes 255 heads, 63 sectors/track, 243147 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System /dev/sda1 1 243147 1953078246 83 Linux
As you can see, /dev/sda1 uses all 243147 cylinders.
Unfortunately, when creating /dev/sdb1, only the first 36585 of 303934 cylinders can be used:
[root@anchor ~]# fdisk /dev/sdb <snip> Command (m for help): p
Disk /dev/sdb: 2499.9 GB, 2499946741760 bytes 255 heads, 63 sectors/track, 303934 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
Command (m for help): n Command action e extended p primary partition (1-4) p Partition number (1-4): 1 First cylinder (1-36585, default 1): Using default value 1 Last cylinder or +size or +sizeM or +sizeK (1-36585, default 36585): Using default value 36585
It appears to me that the 3Ware card is reporting the information correctly, but fdisk is having the trouble. Is this due to running 32-bit software? Do you think running on 64-bit hardware and 64-bit CentOS will fix this problem?
Many thanks. -Sean
On Mon, 2005-08-22 at 17:46, centos-bounces@centos.org wrote:
On Mon, 2005-08-22 at 17:35 -0500, Sean Staats wrote:
We recently bought a 32-bit Xeon system with a 12-port 3Ware RAID card and a dozen 500GB drives. We wanted to create 4TB drive arrays; however, we soon discovered that there is about a 2.2TB drive array size limit on 32-bit hardware. Does that sound correct?
Yes, 2^40 = 2TiB ~ 2.2TB (2.2 * 10^12). This is a PC geometry issue, although Linux can get around it.
Would replacing the 32-bit mobo/cpu with a 64-bit mobo/cpu allow us to use drive arrays larger than 2.2TB?
Actually it's a 3Ware question because 3Ware has an intelligent ASIC on- board. It's driving the disk array, not Linux. It's merely presenting the disk array as a block, and Linux talks to the ASIC, not the disks.
On Tue, 23 Aug 2005 at 10:38am, Sean Staats wrote
It appears to me that the 3Ware card is reporting the information correctly, but fdisk is having the trouble. Is this due to running 32-bit software? Do you think running on 64-bit hardware and 64-bit CentOS will fix this problem?
No. fdisk can't handle disks bigger than 2TiB. You have to use parted. In addition, you can't use msdos (the default) disk labels on disks bigger than 2TiB -- you must use gpt disk labels. This means that can't boot from a >2TiB disk, as neither grub nor lilo understand gpt disk labels.
On 8/23/05, Joshua Baker-LePain jlb17@duke.edu wrote:
On Tue, 23 Aug 2005 at 10:38am, Sean Staats wrote
It appears to me that the 3Ware card is reporting the information correctly, but fdisk is having the trouble. Is this due to running 32-bit software? Do you think running on 64-bit hardware and 64-bit CentOS will fix this problem?
No. fdisk can't handle disks bigger than 2TiB. You have to use parted. In addition, you can't use msdos (the default) disk labels on disks bigger than 2TiB -- you must use gpt disk labels. This means that can't boot from a >2TiB disk, as neither grub nor lilo understand gpt disk labels.
-- Joshua Baker-LePain Department of Biomedical Engineering Duke University
Again - http://www.3ware.com/kb/article.aspx?id=11920
--
Cletus Murphy cletus.murphy@gmail.com wrote:
If you don't see a recurring theme from Cletus, _always_ check 3Ware's knowledge base. 3Ware is the authority on how their cards work. Unlike FRAID, where Linux controlls the drives, Linux _never_ touches the disks with 3Ware -- it's always talking to the on-board intelligence in 3Ware's ASIC running its own firmware (and embedded OS).
As a secondary option, Bugzilla for RHEL3/4 is always a consideration for CentOS 3/4.
Joshua Baker-LePain jlb17@duke.edu wrote:
No. fdisk can't handle disks bigger than 2TiB. You have to use parted. In addition, you can't use msdos (the default) disk labels on disks bigger than 2TiB -- you must use gpt disk labels. This means that can't boot from a
2TiB disk, as neither grub nor lilo understand gpt
disk labels.
I thought LILO just dumbly booted an absolute sector offset? Or is the lack of a MBR in GPT Disk Labels?
With that said -- since I've found *0* info elsewhere -- does anyone know the status of getting Microsoft NT5 (2000+) Logical Disk Manager (LDM)** disk label aka "Dynamic Disc" support in GRUB/Parted?
The kernel has the disk label support for LDM. LDM also looks like a legacy BIOS/DOS disk label aka "Basic Disc" with a legacy MBR and Type 42 slice, so that solves that problem. LILO is just a can boot a absolute sector in the LDM. But there is no LDM support in Parted for slicing or GRUB for booting.
LDM support would solve, once and for all, all those nasty geometry and other dual-boot issues. Especially for Windows XP SP2+ as Microsoft has been changing its geometry and using portions of the MBR for legacy BIOS/DOS disks to store information that would normally go in the defined LDM stores.
-- Bryan
**NOTE: I haven't build a Windows XP/2003 system with anything more than a 2.2TB (2TiB) volume/filesystem, so correct me if LDM doesn't support >2.2TB (>2TiB) without being in something like a GPT disk label. I know the GPT disk label is used on IA-64, and then LDM in it, but I assume Microsoft would have made LDM >2.2TB supporting since it just came out in 1999 (NT5/2000).
-
Bryan J. Smith -
Cletus Murphy -
Joshua Baker-LePain -
Sean Staats