[CentOS] preferred software RAID 10?

Ross S. W. Walker RWalker at medallion.com
Thu Jul 17 23:20:54 UTC 2008

Rudi Ahlers wrote:
> Ross S. W. Walker wrote:
> > Rudi Ahlers wrote:
> >   
> >> John R Pierce wrote:
> >>     
> >>> Rudi Ahlers wrote:
> >>>       
> >>>> And then, how do I setup the partitioning? Do I setup /boot on a 
> >>>> separate RAID "partition"? If so, what happens if I want to replace 
> >>>> the 1st 2 HDD's with bigger ones?
> >>>>         
> >>> each partition is raided seperately with mdadm.... you could make the 
> >>> whole thing one LVM partition thats raid10, then use LVM to dice it up 
> >>> into file systems.
> >>>
> >>> if you have 4 drives and are doing software raid10, you won't be 
> >>> swapping drives with different sizes without a WHOLE lotta pain.
> >>>       
> >> Ok, so how do I do this? Let's say I have 4x 160GB HDD's now, and plan 
> >> on replacing them with 4x 500GB HDD's in the future?
> >>     
> >
> > Personally I would never put an OS install on a higher RAID then RAID1,
> > because it gets too messy to upgrade like you suggested.
> > 
> So you're suggesting that I keep the OS separate from the data? But what 
> happens if both the 1st 2 drives with the OS fails, or needs to be replaced?

Well I was talking 2 separate spindles for the OS, but I guess you got
the idea from my later question.

> >   
> >> What setup would help with a upgrade in the future?
> >>
> >>     
> >>> /boot shouldn't be mirrored, as the BIOS won't know how to boot it. 
> >>> leave /dev/sdb1 the same size as /dev/sda1 and call it /boot2 and try 
> >>> to remember to copy /boot to /boot2 each time you update the kernel.
> >>>       
> >> I understand this, but how do you boot from /boot2 on the 
> >> second HDD if the 1st have failed?
> >>     
> >
> > Could you not get a system that had 2 drives for the OS and 4 drives
> > for data?
> >
> nope, unfortunately not. It's a 2U rackmount chassis with space for only 
> 4 HDD's. I have been thinking about installing the OS onto a USB memory 
> stick, but have never actually got as far as trying to figure out how 
> todo it.

Yeah, problem with USB memory stick is swap on the slow USB will IO wait
the whole box and what if some wise guy comes and says "Oh look someone
forgot a USB memory stick"?

> > I have setup 4 disk RAID10 systems before, but they were never
> > intended to be upgraded (in place at least).
> >
> > I can forward a couple of recipes, but let me first say that to do
> > it from the CentOS install media requires 2 RAID1s and LVM striping
> > because the RAID10 option isn't on the media, but it is functionally
> > equivalent both in useable space and performance.
> >
> Please share your recipes, I'd like to give it a try  :)

Ok, well let me start with the first using 2 RAID1 PVs in a VG and

This will require 2 major steps, one to setup and install and the
other after installation to create striped OS LV because the
installer doesn't let you give options on the LV creation to
make it interleaved.

1) Create 100MB or 256MB primary parts on each disk as type

2) Add those 4 partitions to a RAID1 set, make first 2 active
and the other 2 spare.

3) Allocate the rest of the drive space on the 4 drives as
partition types of MD RAID

4) Create 2 RAID1s, one out of first 2 drives, the other
out of the second 2 drives. Make them of type LVM.

5) Create LVM volume group vg0 out of the 2 PVs.

6) Create 2 4 GB LVs in the VG, 1 called swap0, the other
called rooti (not a typo cause after boot we will create
a root).

7) Install into rooti and reboot

8) After reboot and yum update, create a LV of say 8GB
that have the option '-i 2' on the lvcreate so it
interleaves the allocation between the two RAID1 PVs,
call it 'root'

9) Do a dump/restore of 'rooti' LV to 'root' LV, for
care do it in single user so data isn't 'influx'.

10) Change fstab and grub.conf swapping rooti for root,
do a 'mkinitrd' for the running kernel and then reboot.

11) Keep in mind older initrd files will still have
the old rooti in them! Maybe best to get rid of those

12) If all works well, do a lvrename of rooti to swap1,
do a mkswap on it and add it to fstab with same priority
as swap0 and then swapon -a and swap will be interleaved.

If you need further explanation on any of those steps
just let me know.

I'll give my off-line receipe after I get home from this
business trip


This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.

More information about the CentOS mailing list