We've been wrestling with this for ... rather longer than I'd care to admit.
Host / initiator systems are a number of real and virtualized CentOS 5.5 boxes. Storage arrays / targets are Dell MD3220i storage arrays.
CentOS is not a Dell-supported configuration, and we've had little helpful advice from Dell. There's been some amount of FUD in that Dell don't seem to know what Dell's own software installation (the md3
Dell doesn't seem to have much OS experience generally.
Their docs are pretty inconsistent. I've noted omissions, terminology differences, and procedural differences among the "Owner's Manual", "Deployment Guide", a professional services "Remote Services Installation Agreement" service description. Some of the multipathing guidance we've had comes from their EqualLogic line of storage servers.
Questions:
1: Is there anyone out there running this configuration and are you satisfied with it?
2: We get a set of error messages on the initator at target login. These appear to be benign, and web research suggests it's the result of a driver configuration issue in trying to send instructions to a
http://comments.gmane.org/gmane.linux.iscsi.open-iscsi/5970
http://bit.ly/gguLl7
MD3000i boxes have 1 controller that can execute IO and one that
cannot execute READ/WRITE IO until a special command is sent. In
older kernels, layers like the partition scanning and udev and
hal would send down IO to those disabled paths and we would see
IO errors like in your link.
In newer kernels we have device handler modules (for MD3000i you
would want scsi_dh_rdac (so do "modprobe scsi_dh_rdac" and then
lsmod to see if it is there)) that will detect if the path is
not active and if so it will either not send the IO or it will
not print a error message since we expect the IO to fail.
We'd prefer *not* seeing spurious I/O errors that we don't have to sift through looking for real storage issues.
3: The MD32xxi series is a dual-controller array with multiple ports on each controller. Multiple targets can be logged into from an initiator, with the pathways aggregated by the Linux multipath (device-mapper-multipath) system. There's very little clear documentation on multipath. In particular, any way to trigger alerting from multipath events / failures, or iscsi session actions, would be helpful. The MD32xxi series only supports reporting from a GUI management utility (MDSM) which would be at best problematic to run in a server environment. The other question is: is multipathing typical of iSCSI configuation? Little of the iSCSI docs I've found discusses multipath configurations at all:
http://www.open-iscsi.org/
http://www.cuddletech.com/articles/iscsi/index.html (good but very dated)
http://www.cyberciti.biz/tips/rhel-centos-fedora-linux-iscsi-howto.html (no mention of multipathing)
4: Dell suggests a shutdown procedure including a flush of multipathing paths ("multipath -F" -- in the "Owner's Manual"):
3 Flush the Device Mapper multipath maps list to remove any old or
modified mappings
# multipath F
Presumably this would go into one of the init scripts -- perhaps the /etc/init.d/multipath script, as part of the "stop" sequence (after the multipath daemon is killed). Anyone done this or know why the practice is recommended?
Based on our experience with Dell I would NOT recommend this configuration for others. But we're stuck with it, and any help in getting things configured would be very helpful and gratefully received.
I'm also hoping to get clearance to release docs we've generated, though that's the subject of some internal negotiation.
--
Dr. Ed Morbius
Chief Scientist
Krell Power Systems Unlimited