[CentOS] iSCSI disk preperation

[Jason Brown]

On 02/07/2011 05:09 PM, Dr. Ed Morbius wrote:
> on 15:19 Mon 07 Feb, Ross Walker (rswwalker at gmail.com) wrote:
>> On Mon, Feb 7, 2011 at 2:36 PM, Dr. Ed Morbius <dredmorbius at gmail.com> wrote:
>>> on 13:56 Mon 07 Feb, Jason Brown (jason.brown at millbrookprinting.com) wrote:
>>>> I am currently going through the process of installing/configuring an
>>>> iSCSI target and cannot find a good write up on how to prepare the disks
>>>> on the server.  I would like to mirror the two disks and present them to
>>>> the client.  Mirroring isn't the question, its how I go about it is the
>>>> problem.  When I partitioned the two drives and mirrored them together,
>>>> then presented them to the client, it showed to the client as a disk out
>>>> no partion on it.  Should I partition the drive again and then lay the
>>>> file system down on top of that?  Or should I delete the partitions on
>>>> the target server and just have sda and sdb mirrored, then when the
>>>> client attaches the disk, then partion it (/dev/sdc1) and write the file
>>>> system.
>>>
>>> What are you using for your iSCSI target (storage array)?
>>>
>>> Generally, RAID management of the storage is managed on the target side.
>>> You'd use the target's native abilities to create/manage RAID arrays to
>>> configure one or more physical disks as desired.  If you're using a
>>> dedicated vendor product, it should offer these capabilities through
>>> some interface or another.
>>>
>>> Presentation of the iSCSI device is as a block storage device to the
>>> initiator (host mounting the array).  That's going to be an
>>> unpartitioned device.  You can either further partition this device or
>>> use it as raw storage.  If you're partitioning it, and using multipath,
>>> you'll have to muck with kpartx to make multipath aware of the
>>> partitions.
>>>
>>> We've elected to skip this locally and create a filesystem on the iSCSI
>>> device directly.
>>>
>>> Creating and mounting filesystems are both generally managed on the
>>> initiator.
>>>
>>> Truth is, there's a lot of flexibility with iSCSI, but not a lot of
>>> guidance as to best practices that I could find.  Vendor docs have
>>> tended to be very poor.  Above is my recommendation, and should
>>> generally work.  Alternate configurations are almost certainly possible,
>>> and may be preferable.
>>
>> If a best practices doc could be handed to you right now, what would
>> you like it to contain?
> 
> <grin>
> 
> I've got about 35 pages of that document sitting on my local HD now.
> Negotiating with management about releasing some of it in some form or
> another.
> 
> It's based on some specific vendor experience (Dell MD3200i), and
> CentOS.  Among the problems: I suspect there's a range of experiences
> and configuration issues.
> 
>> I would suspect that it would be different whether your setting up an
>> initiator or a target, so maybe start by splitting it into two
>> sections.
> 
> Absolutely -- if you're setting up your own target, you've got a set of
> problems (and opportunities) not facing those with vendor-supplied kit
> with its various capabilities and limitations.
> 
> 
> Basics:
> 
>   - Terminology.  I found the Wikipedia article and the Open-iSCSI
>     README to be particularly good here.
> 
>   - Presentation of the devices.  It took us a while to realize that we
>     were going to get both a set of raw SCSI devices, AND a set of
>     multipath (/dev/dm-<number>) devices, and that it was the latter we
>     wanted to play with.  We spent a rediculous amount of time trying to
>     debug and understand what turned out to be expected and correct
>     behavior, though this was either undocumented or very unclearly
>     documented.  How much of this is specific to the MD3xxxi kit with
>     its multiple ports and controllers isn't clear to me.
> 
>   - Basic components.  For us these were vendor tools (and identifying
>     which of these were relevant was itself non-trivial), iscsiadm,
>     multipath, and the CentOS netfs config scripts / environment.
>     The device-mapper-mulitpath docs are shite.
> 
>   - Overview of target and initiator setup:  phyisical, cabling, network
>     topology (a dedicated switched LAN or vLAN being recommended, away
>     from core network traffic).
> 
>   - As appropriate: multipath, its role, where it is/isn't needed, what
>     it provides.
> 
>   - Target-side set-up, including virtual disk creation, RAID
>     configuration, network configuration, host-to-LUN mapping, IQN
>     generation / identification, CHAP configuration (if opted), etc.
> 
>     I'd slice this into Linux/CentOS target configuration (specific
>     tasks), preceded by a more generic "things you'll want to take care
>     of" section.  If people/vendors want to fill in their specific
>     configuration procedures in additional sub-sections, that would be
>     an option.
> 
>   - Initiator-side set-up:  installation of base packages, verifying
>     iscsi functionality, verifying multipath functionality, verifying
>     netfs functionality.  Preparing storage (partitioning, filesystem
>     creation).
> 
>   - Target discovery from initiator.  CHAP configuration (if opted),
>     session log-in, state querying, log out.  DiscoveryDB querying,
>     update, manipulation.
> 
>   - Configuring/disabling persistent / on-login iscsi sessions.
> 
>   - Not applicable to us, but booting to an iscsi-mounted root
>     filesystem.
> 
>   - Log / dmesg error/informative messages, interpretation, debugging.
> 
>   - What to do when things go wrong.  An all too infrequent
>     documentation section.
> 
>   - Monitoring and assessment.  "How can I tell if I'm properly
>     configured", target/initiator-side error/issue reporting.  Good ways
>     to tie into existing reporting systems/regimes (SNMP, Nagios, Cacti,
>     ...).
> 
>   - Peformance issues.
> 
>   - Security.  Best I can tell this divides into:
>     - Session authentication (CHAT/CHAP)
>     - Target/initiator host security (specific to protocols used to
>       access either).
>     - Physical and network security -- not necessarily related, but the
>       point being that iSCSI traffic and hardware should generally be
>       isolated.  I'm not aware of encrypted network transports, though I
>       suspect some sort of socket forwarding or IPSEC could be layered
>       in.
>     - Fileystem encryption.  I'd suspect this would be managed
>       client-side, though implications on network datastreams should
>       probably be noted.  Hrm... if it _is_ handled client-side, the
>       network traffic should by ciphertext, not clear.
> 
> If nothing else, this should make a good framework for developing useful
> docs -- the Christmas tree from which to hang the ornaments.  It's
> rather much how my own document started -- unanswered questions /
> unclear elements of the configuration, filled in by research and
> experience.
> 
>  
>> I would be happy to draft something up and put it on a wiki somewhere,
>> but I would need a list of talking points to start with.
>  
> How's this do you?
> 


In our configuration, we are going to have our iSCSI targets and
initiators all connected to the same layer 3 switch and isolate the
iSCSI traffic on separate networks.  Would it be beneficial to also set
up multipath for this as well?