[CentOS] add on sata card relabeling drives, installation

Fri Sep 30 20:20:50 UTC 2011
Lamar Owen <lowen at pari.edu>

On Friday, September 30, 2011 03:36:50 PM Bob Hoffman wrote:
> Below is the issue I am talking about. The system ignores the sda/sdb
> etc labeling to use UUID and things like hd0, hd1...
> Yet mdstat shows you the drives in the useless labeling way...sda sdab
...

Useless?  Those names are what the currently running kernel sees, and thus correspond to the current devices.  You then have to do a little more digging with other tools, but that is what it is.  The output from a cat of /proc/mdstat is raw data, and is more useful for rebuild status than anything else.

> taking out a dead drive and adding a new one negates a real ability to 
> re-label them as you want since the new drive would have a different 
> UUID/serial, etc..and screw up any label system you might have done.

For software RAID, yes.  It makes it a challenge keeping up with what device is what.  However, palimpsest (the GNOME Disk Utility included in EL 6) shows more detail about RAID components; also, if you check the drive itself there is a link in the page to the array device (Go to array).  

It's not quite as easy as using, say, EMC Unisphere to manage arrays, but it's not bad.  EMC arrays show you on the array itself, using an amber LED, exactly which drive is faulted, and also lights the amber fault LED on the enclosure, and numbers the drives by backend bus number, enclosure number, and drive number, so that, say, drive 14 in enclosure 7 on bus 3 would be device 3_7_14 (zero origin on all these numbers).  But when you can have up to 960 drives in a single array such 'luxuries' become necessities (960 drives in a CX4-960; we have a CX4-480 that only will allow 480 drives, currently populated with 230).  I can't imagine managing 480 drives on a Linux box with the current device naming.

> That mdstat is not the only monitoring software that pays attention to 
> the 'useless' labels like sda sdb sdc, etc...

/proc/mdstat?  It's not monitoring software; it's a kernel API mounted on the /proc filesystem; those names can and will change if disk enumeration order changes.  File a bug report with the kernel developers to see this changed.

> I am going to just leave it alone, but I feel their is something missing 
> in the whole theory...
> if the labels sda, sdb, etc mean nothing to the system admin, then why 
> are they used in monitoring software...and worse, why are they the only 
> drive identification listed in the installer???

Hmm, are they?  I'll have to do another install to double check, but I think the drive model numbers/types are listed, but it has been a little bit since I did the install (of my RHEL 6.1 box at least).

If you want the disks listed differently in the installer, the right thing to do is file a bug report upstream (with Red Hat, in other words) on bugzilla.redhat.com, as CentOS is 100% going to do what upstream does in this regard.

> And the disk manager, graphical one, in centos 6 does not list the UUID, 
> it calls it 'worldwide ID'.

That's because it is the WWN in the case of fibre channel, and a unique ID otherwise.  WWN is unique, for fibre channel.  Fibre channel: the original serial-attached SCSI.  And, well, FC is giving way to dual-attach SAS in the enterprise space; 2.5 inch enterprise SAS drives in the EMC VNX series, for instance.

I have found palimpsest (the EL6 'Disk Utility' in the Applications -> System Tools menu) to give a lot of good information in one very nice display.  It doesn't yet deal well with LVM; for that you use system-config-lvm.  But the serial number, the filesystem label, RAID components, etc all in one place.  Also, for one of the controllers I have it shows which port on the card a particular disk is attached to.

While many around this list detest GUI's on servers, I'll be honest; this is one very handy and useful utility that works quite well using ssh tunnelled X.  A similar text-mode interface tool would be super nice, but someone has to write it, it's not in there right now.

> I am sure it is needed, but I still see UDEV using unchanging labeling 
> (things I would use to add grub on a drive) like 'HD0,HD1, etc.

The grub device names are the BIOS names; the grub hd0 should be the BIOS boot drive.  I say 'should be' because I have had a mismatch there before.  So in my example (where /boot is on /dev/sdag) it just so happens that /dev/sdag is grub's hd0, but once the kernel comes up and enumerates the disks what was first shall become last... (hmmm, seems like a quote from the world's most popular book...).

> why even use the labels, just use HD1, HD2 and the UUID and let it be.

The /dev/sdX names are older than udev.  Historical, (or hysterical, depending upon your PoV).  HD1, HD2, etc would be far worse. 

In the general case with PC hardware this is where things stand.  With general PC hardware it becomes the responsibility of the individual systems administrator to simply know the machine inside and out, and document the machine thoroughly.