on 15:19 Mon 07 Feb, Ross Walker (rswwalker at gmail.com) wrote: > On Mon, Feb 7, 2011 at 2:36 PM, Dr. Ed Morbius <dredmorbius at gmail.com> wrote: > > on 13:56 Mon 07 Feb, Jason Brown (jason.brown at millbrookprinting.com) wrote: > >> I am currently going through the process of installing/configuring an > >> iSCSI target and cannot find a good write up on how to prepare the disks > >> on the server. I would like to mirror the two disks and present them to > >> the client. Mirroring isn't the question, its how I go about it is the > >> problem. When I partitioned the two drives and mirrored them together, > >> then presented them to the client, it showed to the client as a disk out > >> no partion on it. Should I partition the drive again and then lay the > >> file system down on top of that? Or should I delete the partitions on > >> the target server and just have sda and sdb mirrored, then when the > >> client attaches the disk, then partion it (/dev/sdc1) and write the file > >> system. > > > > What are you using for your iSCSI target (storage array)? > > > > Generally, RAID management of the storage is managed on the target side. > > You'd use the target's native abilities to create/manage RAID arrays to > > configure one or more physical disks as desired. If you're using a > > dedicated vendor product, it should offer these capabilities through > > some interface or another. > > > > Presentation of the iSCSI device is as a block storage device to the > > initiator (host mounting the array). That's going to be an > > unpartitioned device. You can either further partition this device or > > use it as raw storage. If you're partitioning it, and using multipath, > > you'll have to muck with kpartx to make multipath aware of the > > partitions. > > > > We've elected to skip this locally and create a filesystem on the iSCSI > > device directly. > > > > Creating and mounting filesystems are both generally managed on the > > initiator. > > > > Truth is, there's a lot of flexibility with iSCSI, but not a lot of > > guidance as to best practices that I could find. Vendor docs have > > tended to be very poor. Above is my recommendation, and should > > generally work. Alternate configurations are almost certainly possible, > > and may be preferable. > > If a best practices doc could be handed to you right now, what would > you like it to contain? <grin> I've got about 35 pages of that document sitting on my local HD now. Negotiating with management about releasing some of it in some form or another. It's based on some specific vendor experience (Dell MD3200i), and CentOS. Among the problems: I suspect there's a range of experiences and configuration issues. > I would suspect that it would be different whether your setting up an > initiator or a target, so maybe start by splitting it into two > sections. Absolutely -- if you're setting up your own target, you've got a set of problems (and opportunities) not facing those with vendor-supplied kit with its various capabilities and limitations. Basics: - Terminology. I found the Wikipedia article and the Open-iSCSI README to be particularly good here. - Presentation of the devices. It took us a while to realize that we were going to get both a set of raw SCSI devices, AND a set of multipath (/dev/dm-<number>) devices, and that it was the latter we wanted to play with. We spent a rediculous amount of time trying to debug and understand what turned out to be expected and correct behavior, though this was either undocumented or very unclearly documented. How much of this is specific to the MD3xxxi kit with its multiple ports and controllers isn't clear to me. - Basic components. For us these were vendor tools (and identifying which of these were relevant was itself non-trivial), iscsiadm, multipath, and the CentOS netfs config scripts / environment. The device-mapper-mulitpath docs are shite. - Overview of target and initiator setup: phyisical, cabling, network topology (a dedicated switched LAN or vLAN being recommended, away from core network traffic). - As appropriate: multipath, its role, where it is/isn't needed, what it provides. - Target-side set-up, including virtual disk creation, RAID configuration, network configuration, host-to-LUN mapping, IQN generation / identification, CHAP configuration (if opted), etc. I'd slice this into Linux/CentOS target configuration (specific tasks), preceded by a more generic "things you'll want to take care of" section. If people/vendors want to fill in their specific configuration procedures in additional sub-sections, that would be an option. - Initiator-side set-up: installation of base packages, verifying iscsi functionality, verifying multipath functionality, verifying netfs functionality. Preparing storage (partitioning, filesystem creation). - Target discovery from initiator. CHAP configuration (if opted), session log-in, state querying, log out. DiscoveryDB querying, update, manipulation. - Configuring/disabling persistent / on-login iscsi sessions. - Not applicable to us, but booting to an iscsi-mounted root filesystem. - Log / dmesg error/informative messages, interpretation, debugging. - What to do when things go wrong. An all too infrequent documentation section. - Monitoring and assessment. "How can I tell if I'm properly configured", target/initiator-side error/issue reporting. Good ways to tie into existing reporting systems/regimes (SNMP, Nagios, Cacti, ...). - Peformance issues. - Security. Best I can tell this divides into: - Session authentication (CHAT/CHAP) - Target/initiator host security (specific to protocols used to access either). - Physical and network security -- not necessarily related, but the point being that iSCSI traffic and hardware should generally be isolated. I'm not aware of encrypted network transports, though I suspect some sort of socket forwarding or IPSEC could be layered in. - Fileystem encryption. I'd suspect this would be managed client-side, though implications on network datastreams should probably be noted. Hrm... if it _is_ handled client-side, the network traffic should by ciphertext, not clear. If nothing else, this should make a good framework for developing useful docs -- the Christmas tree from which to hang the ornaments. It's rather much how my own document started -- unanswered questions / unclear elements of the configuration, filled in by research and experience. > I would be happy to draft something up and put it on a wiki somewhere, > but I would need a list of talking points to start with. How's this do you? -- Dr. Ed Morbius Chief Scientist When you need power Krell Power Systems Unlimited Go to Krell!