Hello list members,
In CentOS-5.5 I'm trying to achieve static assignment of SCSI device names for a bunch of RAID-60 drives on a Supermicro motherboard. The "scsi_id" command identifies all drives ok.
The board has one SATA controller and three SAS/SATA controllers ... standard on-board ICH-10 ATA channels, an on-board LSI SAS/SATA controller, and two add-on SAS/SATA contoller cards. There are 13 drives in all, spread across the four controllers, all configured for Linux software RAID.
The problem is in management of the drive names and figuring out which drive to pull in case of failure. Unfortunately the BIOS scan detects only the three drives connected to the ICH-10 SATA controller. That's ok because that's where the RAID-1 boot drives are. However, when the kernel starts it assigns those drives last, not first.
For this reason I want to use a set of udev rules to assign specific names to the drives plugged into specific ports (to maintain my sanity :-) ).
Identifying drives by their ID string (which includes the drive's serial number) and assigning names in the rules works ok. BUT, what happens when I have to swap out a failed drive? The serial number (and possibly model number) changes, and the udev assignment should fail, probably assigning an unexpected /dev/sd? name. RAID rebuild would choke until I change the MD device assignment.
Is it possible to assign SCSI drive names by hardware path instead? I especially want the three RAID1+spare boot drives to always be assigned sda/sdb/sdc, because that sorts out other issues I'm having in CentOS-5.
In the udev rules file I tried piping the output of "scsi_id -g -i -u -s /block/..." through "cut" to extract the path, but I get no match string when I run "udevtest" against that block device. Does the "PROGRAM==....." clause not recognize the pipe symbol? I tried a little shellscript to provide the RESULT match string, but udevtest didn't like that.
Is there a supported way to predictably assign a drive name according to the hardware port it's plugged into ... it would make swapping drives a lot easier, since it becomes 'drive-id-string' agnostic. Better yet, is there any way to tell the kernel the order in which to scan the controllers?
I'm also hoping the problem doesn't radically change when I install CentOS-6 on this box. I'm using CentOS-5 just to get practice in using KVM and RAID-60.
Thanks for any advice you can offer.
Chuck
On 1/27/2011 2:41 PM, Chuck Munro wrote:
Identifying drives by their ID string (which includes the drive's serial number) and assigning names in the rules works ok. BUT, what happens when I have to swap out a failed drive? The serial number (and possibly model number) changes, and the udev assignment should fail, probably assigning an unexpected /dev/sd? name. RAID rebuild would choke until I change the MD device assignment.
If you can figure things out for the initial md device creation, subsequent assembly uses the uuid to match up the members and doesn't care if the disks have been moved around either physically or by detection order. And if you are hot-swapping drives and rebuilding manually, you should be able to find the just-assigned name with 'dmesg'.
on 12:41 Thu 27 Jan, Chuck Munro (chuckm@seafoam.net) wrote:
Hello list members,
In CentOS-5.5 I'm trying to achieve static assignment of SCSI device names for a bunch of RAID-60 drives on a Supermicro motherboard. The "scsi_id" command identifies all drives ok.
The board has one SATA controller and three SAS/SATA controllers ... standard on-board ICH-10 ATA channels, an on-board LSI SAS/SATA controller, and two add-on SAS/SATA contoller cards. There are 13 drives in all, spread across the four controllers, all configured for Linux software RAID.
The problem is in management of the drive names and figuring out which drive to pull in case of failure. Unfortunately the BIOS scan detects only the three drives connected to the ICH-10 SATA controller. That's ok because that's where the RAID-1 boot drives are. However, when the kernel starts it assigns those drives last, not first.
For this reason I want to use a set of udev rules to assign specific names to the drives plugged into specific ports (to maintain my sanity :-) ).
Identifying drives by their ID string (which includes the drive's serial number) and assigning names in the rules works ok. BUT, what happens when I have to swap out a failed drive? The serial number (and possibly model number) changes, and the udev assignment should fail, probably assigning an unexpected /dev/sd? name. RAID rebuild would choke until I change the MD device assignment.
Is it possible to assign SCSI drive names by hardware path instead? I especially want the three RAID1+spare boot drives to always be assigned sda/sdb/sdc, because that sorts out other issues I'm having in CentOS-5.
In the udev rules file I tried piping the output of "scsi_id -g -i -u -s /block/..." through "cut" to extract the path, but I get no match string when I run "udevtest" against that block device. Does the "PROGRAM==....." clause not recognize the pipe symbol? I tried a little shellscript to provide the RESULT match string, but udevtest didn't like that.
Is there a supported way to predictably assign a drive name according to the hardware port it's plugged into ... it would make swapping drives a lot easier, since it becomes 'drive-id-string' agnostic. Better yet, is there any way to tell the kernel the order in which to scan the controllers?
I'm also hoping the problem doesn't radically change when I install CentOS-6 on this box. I'm using CentOS-5 just to get practice in using KVM and RAID-60.
Though I don't swear to understand it well, it's possible that multipath (device-mapper-multipath) may work in your situation. I've been using it for iSCSI storage, where it provides multipathing capabilities, including performance improvements, HA, and persistent device naming.
Whether this applies to hotplugged SCSI devices I'm not so sure, and udev would be my first choice.
The multipath documentation is unfortunately atrocious.
-
Chuck Munro -
Dr. Ed Morbius -
Les Mikesell