Hi,
I just received a new server (HP DL180G5) with 12x 1TB HDs and I bumped into fdisks 2TB limits... Since this is an entry level server, I can't use the classic HP bootable utilities to create smaller volumes et can only create a big RAID6. I found out that: using parted, labelling it gpt and creating the partitions would do the trick. But, what about grub? I read that it does not support gpt... Or is there another way to do it? By example, do you think I could boot on a live CentOS, install HP RAID tools on it (is it possible?) and then create volumes...?
Thx, JD
I have this problem, I just boot from usb sticks?
A 512MB usb pen drive is (just about extinct!) only about £2.00 so get three, make one an ms-dos disk with parted and format it ready. Then pop-in CentOS and install it mounting the usb pen as /boot so grub boots from that msdos partition and then use dd to copy the usb pendrive to two more to have them knocking around (take one home and put it in your sock draw or something) and just keep the image for the pen drive somewhere like with your work files?
2009/1/27 John Doe jdmls@yahoo.com:
Hi,
I just received a new server (HP DL180G5) with 12x 1TB HDs and I bumped into fdisks 2TB limits... Since this is an entry level server, I can't use the classic HP bootable utilities to create smaller volumes et can only create a big RAID6. I found out that: using parted, labelling it gpt and creating the partitions would do the trick. But, what about grub? I read that it does not support gpt... Or is there another way to do it? By example, do you think I could boot on a live CentOS, install HP RAID tools on it (is it possible?) and then create volumes...?
Thx, JD
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
From: James Bensley jwbensley@gmail.com
I have this problem, I just boot from usb sticks? A 512MB usb pen drive is (just about extinct!) only about £2.00 so get three, make one an ms-dos disk with parted and format it ready. Then pop-in CentOS and install it mounting the usb pen as /boot so grub boots from that msdos partition and then use dd to copy the usb pendrive to two more to have them knocking around (take one home and put it in your sock draw or something) and just keep the image for the pen drive somewhere like with your work files?
I guess that would work but I don't think my boss will like having a usb key plugged in each servers...
From: Morten Torstensen morten@mortent.org
I would create two raid logical volumes, one for centos (say, 20GB to 100GB) and one with the rest of the space. Install centos and normal MBR on /dev/sda and then use lvm on the /dev/sdb directly with no partition table needed. I would also strongly consider having two disks mirrored for the system in one lvm vg and the rest in another, but with 1TB disks it is kind of wasted space. Tho with 12 disks you can have 2 disks for RAID1, then 8+P+1 in RAID5.
Yes but, as I said, I sadly don't have access (yet)to vendor utilities that would allow me to create logical volumes... For this entry model of server, there is no offline/bootable utility to access extended RAID configuration (even if HP shipped the utility CD for this unsupported server ^_^) . The BIOS raid setup is very basic, select drives, RAID level, and click... I could do 2 HDs in RAID1, then 10 disks in RAID6, but that would indeed "waste" 1TB.
The best solution would be to be able somehow to run the avanced RAID utility... For that, I need a running OS where I could install it.
Thx, JD
John Doe wrote:
The best solution would be to be able somehow to run the avanced RAID utility... For that, I need a running OS where I could install it.
They don't have a bootable CD image with the tools? I am only familiar with the IBM ServeRAID adapters where you have such a tool.
Another option is of course to create a number of 2TB partitions and create a VG with all of them as PVs. Of course, you would need 6 partitions for this, but once you have them addded to a VG it is pretty seamless.
John Doe wrote:
I just received a new server (HP DL180G5) with 12x 1TB HDs and I bumped into fdisks 2TB limits...
I would create two raid logical volumes, one for centos (say, 20GB to 100GB) and one with the rest of the space. Install centos and normal MBR on /dev/sda and then use lvm on the /dev/sdb directly with no partition table needed.
I would also strongly consider having two disks mirrored for the system in one lvm vg and the rest in another, but with 1TB disks it is kind of wasted space. Tho with 12 disks you can have 2 disks for RAID1, then 8+P+1 in RAID5.
On Tuesday 27 January 2009, Morten Torstensen wrote:
John Doe wrote:
I just received a new server (HP DL180G5) with 12x 1TB HDs and I bumped into fdisks 2TB limits...
I would create two raid logical volumes, one for centos (say, 20GB to 100GB) and one with the rest of the space. Install centos and normal MBR on /dev/sda and then use lvm on the /dev/sdb directly with no partition table needed.
Smartarray controllers user cciss.ko which is one of the few block I/O thingys that doesn't use the linux scsi layer. Devices will be /dev/cciss/c0d0, c0d1, ...
I would also strongly consider having two disks mirrored for the system in one lvm vg and the rest in another, but with 1TB disks it is kind of wasted space. Tho with 12 disks you can have 2 disks for RAID1, then 8+P+1 in RAID5.
Smartarray controllers don't work like "normal" raid. You can have the first part of all your 12 drives be a logical raid1+0 and then the rest be a raid6. In this way you won't "waste" two drive-slots for OS.
/Peter
On Tuesday 27 January 2009, John Doe wrote:
Hi,
I just received a new server (HP DL180G5) with 12x 1TB HDs and I bumped into fdisks 2TB limits... Since this is an entry level server, I can't use the classic HP bootable utilities to create smaller volumes et can only create a big RAID6.
If it does raid6 then you have a p400 or p800 controller, right? If so then hpacucli or similar can easily give you a small logical drive for the OS and then a large one for data.
On the small one partition normally and install grub.
On the big one do one of:
1) partition with parted and gpt 2) use lvm and put a pv directly on the cciss device 3) put the filesystem directly on the cciss device
Also since you have that hardware. Make sure your controller firmware is at 5.22, drives at HPG6 and backplane at 2.00
Cheers, Peter
I found out that: using parted, labelling it gpt and creating the partitions would do the trick. But, what about grub? I read that it does not support gpt... Or is there another way to do it? By example, do you think I could boot on a live CentOS, install HP RAID tools on it (is it possible?) and then create volumes...?
Thx, JD
From: Peter Kjellstrom cap@nsc.liu.se
If it does raid6 then you have a p400 or p800 controller, right? If so then hpacucli or similar can easily give you a small logical drive for the OS and then a large one for data.
Yes, that's the plan but, the thing is to be able to run the utilities... I need either to make a live CD with the HP tools installed, or a "temporary OS with the tools... First try will be to create a RAID6 on 3 disks (=1 TB, so no grub problems), install the OS, run HP ACU, extend the RAID to the 12 disks, and create the logical disks... If first try fails, second try would be to use a temporary USB disk to install a temporary OS. I even thought of Installing the P800 in a model 3xx, while leaving the disks in the DL180 (if cable length permits it), and boot with SmartStart on the 3xx... ^_^ As for the logical disks sizes, we would go with something like 5 disks of 1.9TB. So, just classic msdos partitions. One thing is for sure, HP tries really hard to make it complicated...
JD
On Wednesday 28 January 2009, John Doe wrote:
From: Peter Kjellstrom cap@nsc.liu.se
If it does raid6 then you have a p400 or p800 controller, right? If so then hpacucli or similar can easily give you a small logical drive for the OS and then a large one for data.
Yes, that's the plan but, the thing is to be able to run the utilities... I need either to make a live CD with the HP tools installed, or a "temporary OS with the tools... First try will be to create a RAID6 on 3 disks (=1 TB, so no grub problems), install the OS, run HP ACU, extend the RAID to the 12 disks, and create the logical disks... If first try fails, second try would be to use a temporary USB disk to install a temporary OS.
This sounds much better to me. Invest 30 min. and install a centos-5 to a 4G usb-stick. Put hpacucli and hpaducli on it and then you can configure, manage and diagnose any server you want.
I even thought of Installing the P800 in a model 3xx, while leaving the disks in the DL180 (if cable length permits it), and boot with SmartStart on the 3xx... ^_^ As for the logical disks sizes, we would go with something like 5 disks of 1.9TB. So, just classic msdos partitions. One thing is for sure, HP tries really hard to make it complicated...
Why would you make 5 logical drives? Why use partition tables?
I think hpacucli works fine but it sure would have been nice to be able to create flexible logical drives in the smartarray bios.
/Peter
Yes, that's the plan but, the thing is to be able to run the utilities... I need either to make a live CD with the HP tools installed, or a "temporary OS with the tools... First try will be to create a RAID6 on 3 disks (=1 TB, so no grub problems), install the OS, run HP ACU, extend the RAID to the 12 disks, and create the logical disks... If first try fails, second try would be to use a temporary USB disk to install a temporary OS.
This sounds much better to me. Invest 30 min. and install a centos-5 to a 4G usb-stick. Put hpacucli and hpaducli on it and then you can configure, manage and diagnose any server you want.
I meant RAID6 on 4 disks of course... ^_^ I could install and boot, but the CLI version of the ACU is a bit intimidating... And long and complex command lines, without command history is really painful. Anyway, I managed and am currently extending the array to the 12 disks... It seems that it is going to take around 3 days! Maybe because the cache battery is not fully charged and so the writes are not cached... After that, I need to reduce the "boot" logical disk down to a dozen of GBs. And then, I need to create the other(s) logical disk(s).
As for the logical disks sizes, we would go with something like 5 disks of 1.9TB. So, just classic msdos partitions. One thing is for sure, HP tries really hard to make it complicated...
Why would you make 5 logical drives? Why use partition tables?
Hum... just wanted to be fdisk friendly (no gpt)... But, since grub problem should be solved, I guess I could make 1 big logical disk with gpt and forget about fdisk...
I saw that the use of LVM was tossed around, don't know if the OP is/plans on using it. If you use ext3 on lvm, you can do a background fsck while the system is up & fs mounted:
I must admit that I have never used lvm... We don't really need resizable/expandable volumes, etc... the server's capacity is already maxed. We try to follow the "KISS" principle as much as we can. But the live background fsck seems nice. Do you really think I should learn lvm rightaway?
JD
On Wednesday 28 January 2009, John Doe wrote:
Yes, that's the plan but, the thing is to be able to run the utilities... I need either to make a live CD with the HP tools installed, or a "temporary OS with the tools... First try will be to create a RAID6 on 3 disks (=1 TB, so no grub problems), install the OS, run HP ACU, extend the RAID to the 12 disks, and create the logical disks... If first try fails, second try would be to use a temporary USB disk to install a temporary OS.
This sounds much better to me. Invest 30 min. and install a centos-5 to a 4G usb-stick. Put hpacucli and hpaducli on it and then you can configure, manage and diagnose any server you want.
I meant RAID6 on 4 disks of course... ^_^ I could install and boot, but the CLI version of the ACU is a bit intimidating... And long and complex command lines, without command history is really painful.
True, that's why you run hpacucli from the linux shell :-)
This way bash (or whatever you use) will save the history.
Anyway, I managed and am currently extending the array to the 12 disks... It seems that it is going to take around 3 days! Maybe because the cache battery is not fully charged and so the writes are not cached... After that, I need to reduce the "boot" logical disk down to a dozen of GBs. And then, I need to create the other(s) logical disk(s).
I think a sane end config is one logical drive for the OS (further chopped up with a ms-dos partition table) and one large logical drive for data (possibly chopped up with a gpt partition table or lvm).
As for the logical disks sizes, we would go with something like 5 disks of 1.9TB. So, just classic msdos partitions. One thing is for sure, HP tries really hard to make it complicated...
Why would you make 5 logical drives? Why use partition tables?
Hum... just wanted to be fdisk friendly (no gpt)...
The is no reason to be ms-dos-partition friendly on the "data device", IMO.
But, since grub problem should be solved, I guess I could make 1 big logical disk with gpt and forget about fdisk...
Don't mix things up here, fdisk is a tool that can only handle ms-dos partition tables. parted is a tool that can handle many types, including gpt. It's the partition table type that is the problem not the tool so to speak.
I saw that the use of LVM was tossed around, don't know if the OP is/plans on using it. If you use ext3 on lvm, you can do a background fsck while the system is up & fs mounted:
I must admit that I have never used lvm... We don't really need resizable/expandable volumes, etc... the server's capacity is already maxed. We try to follow the "KISS" principle as much as we can.
You can look at LVM as just another way to partition a drive. In that sense it's more flexible and harder to mess up than using, say, parted+gpt.
With parted+gpt you'd partition the /dev/cciss/cXdY device into many cXdYpN devices. With lvm you'd get to name them and they would be available as /dev/VOLUMEGROUPNAME/LOGICALVOLUMENAME.
Go with the way you know and understand.
But the live background fsck seems nice.
This feature seems to violate the KISS principle for sure...
/Peter
Do you really think I should learn lvm rightaway?
JD
Anyway, I managed and am currently extending the array to the 12 disks... It seems that it is going to take around 3 days! Maybe because the cache battery is not fully charged and so the writes are not cached... After that, I need to reduce the "boot" logical disk down to a dozen of GBs. And then, I need to create the other(s) logical disk(s).
FYI, I found out hpacucli cannot reduce the size of a logical disk; even if there is 99% of unpartioned space... :/ So, since my boss wants a clean install, I started over and chose option 2: a CentOS external USB disk + hpacucli installed on it...
JD
On Tue, Jan 27, 2009 at 7:20 AM, John Doe jdmls@yahoo.com wrote:
Hi,
I just received a new server (HP DL180G5) with 12x 1TB HDs and I bumped into fdisks 2TB limits... Since this is an entry level server, I can't use the classic HP bootable utilities to create smaller volumes et can only create a big RAID6. I found out that: using parted, labelling it gpt and creating the partitions would do the trick. But, what about grub? I read that it does not support gpt... Or is there another way to do it? By example, do you think I could boot on a live CentOS, install HP RAID tools on it (is it possible?) and then create volumes...?
Thx, JD
I came across this article you may find useful: http://www.unixgods.org/~tilo/linux_larger_2TB.html
I should say that I STRONGLY recommend not creating ext3 file systems in the 2TB+ range - fsck takes too long and you'd hate to get hit by one of those in what is supposed to be a "quick" reboot...and disabling them on the file system isn't a good idea either.
On Tue, 27 Jan 2009 at 6:43pm, Jake wrote
I should say that I STRONGLY recommend not creating ext3 file systems in the 2TB+ range - fsck takes too long and you'd hate to get hit by one of those in what is supposed to be a "quick" reboot...and disabling them on the file system isn't a good idea either.
On the other hand, nothing is as well supported on RHEL/CentOS as is ext3. So if you're data is really important to you, think hard about using another FS.
On Jan 27, 2009, at 6:46 PM, Joshua Baker-LePain jlb17@duke.edu wrote:
On Tue, 27 Jan 2009 at 6:43pm, Jake wrote
I should say that I STRONGLY recommend not creating ext3 file systems in the 2TB+ range - fsck takes too long and you'd hate to get hit by one of those in what is supposed to be a "quick" reboot...and disabling them on the file system isn't a good idea either.
On the other hand, nothing is as well supported on RHEL/CentOS as is ext3. So if you're data is really important to you, think hard about using another FS.
This is sound advice.
Best way is probably to use lvm and create multiple 2TB LVs with ext3. Disable auto-fsck and look at doing proactive online fsck. There was a post about a month ago on performing online fscks with snapshots.
For that kind of storage I might look at Solaris and ZFS though.
-Ross
On Tue, 2009-01-27 at 18:46 -0500, Joshua Baker-LePain wrote:
On Tue, 27 Jan 2009 at 6:43pm, Jake wrote
I should say that I STRONGLY recommend not creating ext3 file systems in the 2TB+ range - fsck takes too long and you'd hate to get hit by one of those in what is supposed to be a "quick" reboot...and disabling them on the file system isn't a good idea either.
On the other hand, nothing is as well supported on RHEL/CentOS as is ext3. So if you're data is really important to you, think hard about using another FS.
Actually, on RHEL, the *only* filesystems that upstream *officially* supports are ext2/3 and GFS. Not XFS, nor reiser, nor JFS. Nada...
Well, maybe FAT for USB-attached storage... ;)
But if you're using CentOS, it's entirely up to you... If I were in RHEL-land (meaning: at a company willing to pony up for licenses), I'd consider a GFS2 cluster shared out via NFS. Or maybe an OCFS2 NFS cluster. If at a company using CentOS, I'd consider an OFCS2/NFS cluster or heartbeat/XFS/NFS. For home? XFS (or JFS if you like). But then, I'm willing (and capable) of supporting the mess I create. It all depends upon one's comfort level with getting out of a jam when one strays out of the "sweet spots" of available help...
-I
On Wednesday 28 January 2009, Jake wrote: ...
I came across this article you may find useful: http://www.unixgods.org/~tilo/linux_larger_2TB.html
I should say that I STRONGLY recommend not creating ext3 file systems in the 2TB+ range
I consider that FUD. We have many ext3 filesystems >2T and the run ok. Sure we do disable automatic fsck on reboot but we schedule a manual fsck when we get the opportunity. IMHO automatic fsck on reboot after, say, 30 boots is a pure desktop/laptop thing. When you have servers that stay up you'll have to plan for fsck anyway.
As Joshua wrote, there is no filesystem more supported and tried on CentOS...
/Peter
- fsck takes too long and you'd hate to get hit by one of
those in what is supposed to be a "quick" reboot...and disabling them on the file system isn't a good idea either.
how to stop "fsck" on boot?
--- 09/1/28 (星期三)，Peter Kjellstrom cap@nsc.liu.se 寫道：
寄件者: Peter Kjellstrom cap@nsc.liu.se 主旨: Re: [CentOS] More than 2TB RAID... 收件者: centos@centos.org 日期: 2009 1 28 星期三 上午 7:40 On Wednesday 28 January 2009, Jake wrote: ...
I came across this article you may find useful: http://www.unixgods.org/~tilo/linux_larger_2TB.html
I should say that I STRONGLY recommend not creating
ext3 file systems in
the 2TB+ range
I consider that FUD. We have many ext3 filesystems >2T and the run ok. Sure we do disable automatic fsck on reboot but we schedule a manual fsck when we get the opportunity. IMHO automatic fsck on reboot after, say, 30 boots is a pure desktop/laptop thing. When you have servers that stay up you'll have to plan for fsck anyway.
As Joshua wrote, there is no filesystem more supported and tried on CentOS...
/Peter
- fsck takes too long and you'd hate to get hit by
one of
those in what is supposed to be a "quick"
reboot...and disabling them on
the file system isn't a good idea either.
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
______________________________________________________________________________________________________ 付費才容量無上限？Yahoo!奇摩電子信箱2.0免費給你，信件永遠不必刪！ http://tw.mg0.mail.yahoo.com/dc/landing
From: mcclnx mcc mcclnx@yahoo.com.tw
how to stop "fsck" on boot?
in fstab: " The sixth field, (fs_passno), is used by the fsck(8) program to determine the order in which filesystem checks are done at reboot time. The root filesystem should be specified with a fs_passno of 1, and other filesystems should have a fs_passno of 2. Filesystems within a drive will be checked sequentially, but filesystems on different drives will be checked at the same time to utilize parallelism available in the hardware. If the sixth field is not present or zero, a value of zero is returned and fsck will assume that the filesystem does not need to be checked."
or maybe tune2fs: "-c max-mount-counts Adjust the number of mounts after which the filesystem will be checked by e2fsck(8). If max-mount-counts is 0 or -1, the number of times the filesystem is mounted will be disregarded by e2fsck(8) and the kernel.
JD
PS: sorry if double post but I got from yahoo: centos@centos.org: No MX or A records for centos.org
Peter Kjellstrom wrote:
On Wednesday 28 January 2009, Jake wrote: ...
I came across this article you may find useful: http://www.unixgods.org/~tilo/linux_larger_2TB.html
I should say that I STRONGLY recommend not creating ext3 file systems in the 2TB+ range
I consider that FUD. We have many ext3 filesystems >2T and the run ok. Sure we do disable automatic fsck on reboot but we schedule a manual fsck when we get the opportunity. IMHO automatic fsck on reboot after, say, 30 boots is a pure desktop/laptop thing. When you have servers that stay up you'll have to plan for fsck anyway.
I saw that the use of LVM was tossed around, don't know if the OP is/plans on using it. If you use ext3 on lvm, you can do a background fsck while the system is up & fs mounted:
http://markmail.org/message/5ipnsva3xkdyzzfy
As Joshua wrote, there is no filesystem more supported and tried on CentOS...
Plus it should be a trivial upgrade to ext4 . . .
-
Ian Forde -
Jake -
James Bensley -
John Doe -
Joshua Baker-LePain -
mcclnx mcc -
Morten Torstensen -
Peter Kjellstrom -
Pintér Tibor -
Ross Walker -
Toby Bluhm