On Mon, Oct 12, 2009 at 06:44:55PM -0700, James A. Peltier wrote:

On Mon, 12 Oct 2009, Eugene Vilensky wrote:

...

Hi List,

Does anyone here use Dell Equallogic IP SAN? If so, do you have any immediate high points / low points you wouldn't mind discussing on- or off-list ?

Yes, it works fine, however, we don't use the snapshot capability of the system as it is not as fine grained as we would like.

Hmm.. what kind of problems do you have with snapshots?

-- Pasi

On Wed, Oct 14, 2009 at 12:36 PM, James A. Peltier jpeltier@fas.sfu.ca wrote:

They take up 16MB  of space regardless of the amount of data that changes. A single bit changes and 16MB is taken up.  This is not tunable.

I know they run Linux but is the internal file system some secret sauce or did they just slap tools on LVM/ext3? Somewhere in between?

James A. Peltier wrote:

Mounting a snapshot requires the generation of a new UUID for the file system in order to be able to use it. Perhaps this is a XFS limitation, but unlike ZFS they aren't immediately available. We need to perform this action extensively to backup file systems and allow for quick data recovery.

Another side effect of snapshots and using LVM for example(applies to any vendor's block based snapshots), is you can't easily take a snapshot of an LVM-based file system, then mount that snapshot on the same system, LVM will bitch. I spent some time trying to play with dynamically changing the UUID in the volume group but gave up, too complicated for my needs, instead I just mount the snapshot on another system(or a VM with either Raw device mapping in VMware or a software iSCSI initiator on the guest).

I've written a few scripts for things like mysql snapshots and Oracle snapshots over the years with my 3PAR array, using SSH and key-based authentication the interface is pretty easy, a single command to create a read-only snapshot, then another command to create a read-write (read-write snapshots must be children of read-only ones in the 3PAR world anyways). The process takes less than a second, and if I want to snapshot a dozen volumes at the same instant, still a single command and the array ensures all volumes are taken at the same point in time, very helpful for Oracle snapshots which can get messy if your dealing with multiple volumes and not having them all perfectly in sync with each other.

nate

John R Pierce wrote:

if you have a SAN that supports logical volumes that have snapshot and grow and such, I'm not sure why you'd bother with LVM on top of that, it would just further obfuscate things.

Makes life easier for me when using MPIO, auto detection of the volume no matter what path it shows up with. And I can slap an easily readable "label" on the volume so I know what it is.

Sample log from one of my oracle snapshot procedures: http://portal.aphroland.org/~aphro/san/oracle-restore-prod-oracle-1a_2008031...

And it allows me to better control growth with thin provisioning, some apps are not thin provisioning friendly so I restrict them with LVM, knowing that I can easily do an online resize at any time without touching the array.

nate

Ray Van Dolson wrote:

Couldn't you accomplish this with the "alias" parameter in your multipath.conf file per LUN? We have this set up here so there's a /dev/mpath/<friendly_name> based on the WWID.

I suppose I could, I recall the friendly name thing a while ago, it seemed more complicated than using LVM though to me at the time. I came up with a system a couple of years ago and haven't had much need to re-visit it.

Another option I think was to use the file system label feature at least with ext3. But I found the other advantages to LVM, so stuck to that, even though I've yet to have more than 1 LV in any particular VG on my SANs.

I came up with this script at the time to mount/unmount my volumes: http://portal.aphroland.org/~aphro/mount_san.init

Mainly to work around limitations in RHEL/CentOS 4 with regards to iSCSI volumes being mounted at the right time, and unmounted at the right time. I believe RHEL/CentOS 5 has fixed this, but I haven't had a need to try out the "fix" since my setup is working well.

nate

And how do you grow a filesystem without LVM? Online?

It certainly is not easy! http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/html/Online_Storag...

Here's hoping for an integrated solution in EL6, but from the looks of Fedora, it's not likely is it?

On Wed, Oct 14, 2009 at 10:52:19AM -0700, nate wrote:

James A. Peltier wrote:

...

Mounting a snapshot requires the generation of a new UUID for the file system in order to be able to use it. Perhaps this is a XFS limitation, but unlike ZFS they aren't immediately available. We need to perform this action extensively to backup file systems and allow for quick data recovery.

Another side effect of snapshots and using LVM for example(applies to any vendor's block based snapshots), is you can't easily take a snapshot of an LVM-based file system, then mount that snapshot on the same system, LVM will bitch. I spent some time trying to play with dynamically changing the UUID in the volume group but gave up, too complicated for my needs, instead I just mount the snapshot on another system(or a VM with either Raw device mapping in VMware or a software iSCSI initiator on the guest).

I've written a few scripts for things like mysql snapshots and Oracle snapshots over the years with my 3PAR array, using SSH and key-based authentication the interface is pretty easy, a single command to create a read-only snapshot, then another command to create a read-write (read-write snapshots must be children of read-only ones in the 3PAR world anyways). The process takes less than a second, and if I want to snapshot a dozen volumes at the same instant, still a single command and the array ensures all volumes are taken at the same point in time, very helpful for Oracle snapshots which can get messy if your dealing with multiple volumes and not having them all perfectly in sync with each other.

On Equallogic you can group your volumes, and snapshot the whole group. So it's meant exactly for this purpose; your database tables, logs and indices (and what not) are quaranteed to be snapshotted at the exact same time.

-- Pasi