[CentOS] who uses Lustre in production with virtual machines?

[Les Mikesell]

On 8/4/2010 10:10 AM, Emmanuel Noobadmin wrote:
>
> derivative work or something along those lines.
>
>> the  OpenSolaris or NexentaStor versions since you wouldn't be using much else
>> from the system anyway.
>
> If I really have to, but I was hoping I wouldn't need to learn another
> relatively similar OS and get myself confused and do something
> catastrophic while in console one day. Especially since I'm way behind
> schedule on picking up another programming language for projects my
> boss wants me to evaluate.

That's sort of the point of nexentastor which gives you a web interface 
to manage the filesystems and sharing since you don't need anything 
else.  But the free community edition only goes to 12 TB.  That might be 
enough per-host if you are going to layer something else on top, though.

>> Snapshots and block-level de-dup are other features of zfs - but I think
>> you'll lose that if you wrap anything else over it.  Maybe you could overcommit an
>> iscsi export expecting the de-dup to make up the size difference and use
>> that as a block level component of something else.
>
> Honestly, I've no idea what all that was about until I go read them up
> later although I understand vaguely from past reading that snapshot is
> like a backup copy

It is good for 2 things - you can snapshot for local 'back-in-time' 
copies without using extra space, and you can do incremental 
dump/restores from local to remote snapshots.

> However, in my ideal configuration, when a VM host server dies, I just
> want to be able to start a new VM instance on a surviving machine
> using the correct VM image/disk file on the network storage and resume
> full functionality.

The VM host side is simple enough if its disk image is intact.  But, if 
you want to survive a disk server failure you need to have that 
replicated which seems like your main problem.

> Since bulk of the actual changes is to "files" in the virtual disk
> file, having snapshot capabilities on the underlying fs doesn't seem
> to be useful. ZFS checksum ensuring that all sectors/inodes of that
> image file are error free seems more critical. Please do point out if
> I am mistaken though!

If you can tolerate a 'slightly behind' backup copy, you could probably 
build it on top of zfs snapshot send/receive replication.   Nexenta has 
some sort of high-availability synchronous replication in their 
commercial product but I don't know the license terms.  The part I 
wonder about in all of these schemes is how long it takes to recover 
when the mirroring is broken.  Even with local md mirrors I find it 
takes most of a day even with < 1Tb drives with other operations 
becoming impractically slow.

-- 
   Les Mikesell
    lesmikesell at gmail.com