I am attempting to get multipath working with device-mapper (CentOS 4.2 and 4.3). It works on EVERY install of mine from RH (also v4.2, 4.3), but the same multipath.conf imported to all my installs of CentOS do not work. Note that I have tested a working 4.2 configuration file from RH on CentOS 4.2 and a working 4.3 configuration (it changed slightly) on CentOS 4.3. Neither worked. Our production systems are RH 4.2 so the 4.2 configuration file is below.
Configuration: Promise vTrak 300i iSCSI iscsi-initiator-utils from 4.2/4.3 device-mapper-multipath from 4.2/4.3
Procedure: 1. Configure iscsi.conf and multipath.conf 2. Start iscsi: service iscsi start 3. Start multipath: service multipathd start 4. Make sure no stale paths exist: multipath -F 5. Scan and create paths: multipath -v 3
Below are the relevant configuration files:
======================================================== My iscsi.conf is very simple: DiscoveryAddress=172.16.160.60
========================================================
From iscsi-ls, the output is:
******************************************************************************* SFNet iSCSI Driver Version ...4:0.1.11(12-Jan-2005) ==================================================== There are two sessions, this creates two targets: # ls /dev/sd* /dev/sda /dev/sda2 /dev/sda5 /dev/sdb /dev/sdb2 /dev/sdb5 /dev/sda1 /dev/sda3 /dev/sda6 /dev/sdb1 /dev/sdb3 /dev/sdb6
===================================================== The multipath.conf is as follows: defaults { multipath_tool "/sbin/multipath -v0" udev_dir /dev polling_interval 10 default_selector "round-robin 0" default_path_grouping_policy multibus default_getuid_callout "/sbin/scsi_id -g -u -s /block/%n" default_prio_callout "/bin/true" default_features "0" rr_wmin_io 100 failback immediate } devnode_blacklist { devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*" devnode "^hd[a-z][[0-9]*]" devnode "^hda[0-9]*" devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]" } multipaths { multipath { wwid 222ea000155eadda4 alias stor path_grouping_policy multibus path_checker readsector0 path_selector "round-robin 0" failback immediate } }
================================================== The output from "multipath -v 3" of my CentOS machine is below: load path identifiers cache # # all paths in cache : # 222ea000155eadda4 0:0:0:0 sda 8:0 [ready] Promise /VTrak M300i /2200 222ea000155eadda4 1:0:0:0 sdb 8:16 [ready] Promise /VTrak M300i /2200 dm-0 blacklisted dm-1 blacklisted hda blacklisted hdc blacklisted md0 blacklisted ram0 blacklisted ram10 blacklisted ram11 blacklisted ram12 blacklisted ram13 blacklisted ram14 blacklisted ram15 blacklisted ram1 blacklisted ram2 blacklisted ram3 blacklisted ram4 blacklisted ram5 blacklisted ram6 blacklisted ram7 blacklisted ram8 blacklisted ram9 blacklisted ===== path info sda (mask 0x1f) ===== bus = 1 dev_t = 8:0 size = 3503903232 vendor = Promise product = VTrak M300i rev = 2200 h:b:t:l = 0:0:0:0 tgt_node_name = serial = 000000000000000000000000D46D0E4B486DC672 path checker = readsector0 (internal default) state = 2 getprio = /bin/true (internal default) prio = 0 uid = 222ea000155eadda4 (cache) ===== path info sdb (mask 0x1f) ===== bus = 1 dev_t = 8:16 size = 3503903232 vendor = Promise product = VTrak M300i rev = 2200 h:b:t:l = 1:0:0:0 tgt_node_name = serial = 000000000000000000000000D46D0E4B486DC672 path checker = readsector0 (internal default) state = 2 getprio = /bin/true (internal default) prio = 0 uid = 222ea000155eadda4 (cache) # # all paths : # 222ea000155eadda4 0:0:0:0 sda 8:0 [ready] Promise /VTrak M300i /2200 222ea000155eadda4 1:0:0:0 sdb 8:16 [ready] Promise /VTrak M300i /2200 pgpolicy = multibus (LUN setting) selector = round-robin 0 (LUN setting) features = 0 (internal default) hwhandler = 0 (internal default) rr_weight = 2 (config file default) no_path_retry = -1 (config file default) pg_timeout = NONE (internal default) 0 3290934 multipath 0 0 1 1 round-robin 0 2 1 8:0 100 8:16 100 set ACT_CREATE: map does not exists message stor 0 switch_group 1
create: stor (222ea000155eadda4) [size=1 GB][features="0"][hwhandler="0"] _ round-robin 0 _ 0:0:0:0 sda 8:0 [ready] _ 1:0:0:0 sdb 8:16 [ready]
reloading features
^^^^^^^^ Notice the 1GB in the map created. I have read in mailing list posts that 1GB is returned when it is unable to find the size.
================================================== The output from "multipath -v 3" of the working 4.2/4.3 machine is below:
hda blacklisted hdc blacklisted md0 blacklisted ram0 blacklisted ram10 blacklisted ram11 blacklisted ram12 blacklisted ram13 blacklisted ram14 blacklisted ram15 blacklisted ram1 blacklisted ram2 blacklisted ram3 blacklisted ram4 blacklisted ram5 blacklisted ram6 blacklisted ram7 blacklisted ram8 blacklisted ram9 blacklisted path sda not found in pathvec
===== path sda ===== bus = 1 dev_t = 8:0 size = 3503903232 vendor = Promise product = VTrak M300i rev = 2200 h:b:t:l = 0:0:0:0 tgt_node_name = serial = 000000000000000000000000D46D0E4B486DC672Ð path checker = readsector0 (internal default) state = 2 getprio = /bin/true (internal default) prio = 0 getuid = /sbin/scsi_id -g -u -s /block/%n (internal default) uid = 222ea000155eadda4 (callout) path sdb not found in pathvec
===== path sdb ===== bus = 1 dev_t = 8:16 size = 3503903232 vendor = Promise product = VTrak M300i rev = 2200 h:b:t:l = 1:0:0:0 tgt_node_name = serial = 000000000000000000000000D46D0E4B486DC672Ð path checker = readsector0 (internal default) state = 2 getprio = /bin/true (internal default) prio = 0 getuid = /sbin/scsi_id -g -u -s /block/%n (internal default) uid = 222ea000155eadda4 (callout) # # all paths : # 222ea000155eadda4 0:0:0:0 sda 8:0 [VTrak M300i ] 222ea000155eadda4 1:0:0:0 sdb 8:16 [VTrak M300i ] pgpolicy = multibus (LUN setting) selector = round-robin 0 (LUN setting) features = 0 (internal default) hwhandler = 0 (internal default) 0 3492951862 multipath 0 0 1 1 round-robin 0 2 1 8:0 1000 8:16 1000 action preset to 0 action set to 4 create: stor (222ea000155eadda4) [size=1 TB][features="0"][hwhandler="0"] _ round-robin 0 _ 0:0:0:0 sda 8:0 _ 1:0:0:0 sdb 8:16
message stor 0 switch_group 1
^^^^^^^^ it detects the size correctly (it is over 1TB, it seems to do some rounding when it reports it here)
====================================
note the multipath config lines: not working: 0 3290934 multipath 0 0 1 1 round-robin 0 2 1 8:0 100 8:16 100 working: 0 3492951862 multipath 0 0 1 1 round-robin 0 2 1 8:0 1000 8:16 1000
The primary difference is in the 2nd number. That is, according to the multipath wiki, the "target length in 512-bytes blocks" for the multipath device. The one that does not work is smaller by a factor of 1063 (close to 1024, maybe it is using different units?)
Has anyone run into any issue of this sort?