[CentOS] HBA enumeration and multipath configuration

Tue Apr 14 00:05:13 UTC 2015
Tom Robinson <tom.robinson at motec.com.au>

# cat /etc/redhat-release
CentOS Linux release 7.1.1503 (Core)
# uname -r
3.10.0-123.20.1.el7.x86_64

Hi,

We use iSCSI over a 10G Ethernet Adapter and SRP over an Infiniband adapter to provide multipathing
to our storage:

# lspci | grep 10-Gigabit
81:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
81:00.1 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection (rev 01)
# lspci | grep Mellanox
06:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE]
(rev b0)

Originally the HBAs came up as host0, host1 and host2. After recent reboots we get host1, host2 and
host3:

# ls -l /sys/class/scsi_host/host*
lrwxrwxrwx. 1 root root 0 Apr  9 19:10 /sys/class/scsi_host/host1 ->
../../devices/platform/host1/scsi_host/host1
lrwxrwxrwx. 1 root root 0 Apr  9 08:12 /sys/class/scsi_host/host2 ->
../../devices/pci0000:00/0000:00:03.2/0000:06:00.0/host2/scsi_host/host2
lrwxrwxrwx. 1 root root 0 Apr  9 08:12 /sys/class/scsi_host/host3 ->
../../devices/pci0000:00/0000:00:03.2/0000:06:00.0/host3/scsi_host/host3

I've observed that at different times during configuration of this host we have had varying
combinations of the mapping of HBA to an enumerated /sys node. e.g.:

host0
host2
host3

or

host1
host3
host4

other combinations, etc.

This messes up our multipath filtering for weightedpath priority settings. From /etc/multipath.conf:

defaults {
        user_friendly_names no
        find_multipaths yes
        prio weightedpath
        prio_args "hbtl 0:.:.:.* 1 [1-9]:.:.:.* 10"
        path_grouping_policy group_by_prio
        path_selector "service-time 0"
        failback immediate
        polling_interval 5
        no_path_retry 12
}

The setting for prio_args fails to set the correct path map topology and priority against an HBA as
I can't predict what HBA is what. Consistently iSCSI claims the lowest number but I can't say if
that's going to be 0 or 1 a this stage.

Does anyone know how to configure HBAs to stay on a fixed enumeration at boot?

Kind regards,
Tom

-- 

Tom Robinson
IT Manager/System Administrator

MoTeC Pty Ltd

121 Merrindale Drive
Croydon South
3136 Victoria
Australia

T: +61 3 9761 5050
F: +61 3 9761 5051   
E: tom.robinson at motec.com.au


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: OpenPGP digital signature
URL: <http://lists.centos.org/pipermail/centos/attachments/20150414/9117a23a/attachment-0005.sig>