[CentOS] LVM performance with snapshots

[Ross S. W. Walker]

> -----Original Message-----
> From: centos-bounces at centos.org 
> [mailto:centos-bounces at centos.org] On Behalf Of t m
> Sent: Tuesday, March 13, 2007 7:41 PM
> To: CentOS mailing list
> Subject: [CentOS] LVM performance with snapshots
> 
> I've been experimenting with LVM and snapshots under Centos 4.4
> recently, and have found performance to be horrible while a snapshot
> of a logical volume exists.
> 
> I expected some performance degradation with a snapshot in place due
> to my understanding of the copy-on-write behavior, but my system
> becomes almost unusable when writing to my system with a snapshot in
> place.  For example, I can write a 4 gigabyte file in about 16 seconds

What speed do you get if you mount the file system with the 'sync'
option?

4GB in 16 seconds equates to 268MB/s, and at 4k block size you either
have a phenomenal fiber channel based RAID system or a whole lot of
memory that provides a huge write-back cache, or both.

> with no snapshots in place, but the same file either takes about 4
> minutes to write, or my systems seems  to hang when a snapshot exists.

Yes, now the LVM copy-on-write is kicking in and for each block
written it now needs to read the existing block, seek to the snapshot,
write the existing block, seek back to position and write the new block.

The fact that the penalty is so high, and the iowait times are high
lets me think that you do not have that massive fiber channel backed
RAID setup.

>  The top and iostat commands seem to show very high iowait times and
> very low actual I/O throuhput, and pdflush seems to be quite busy
> doing something.  Removing the snapshot immediately brought
> performance back to normal.

pdflush is writing out write-cache to disk. The disks are taking so
long to service the requests now that the write cache is backing up.

> Are LVM snapshots not recommended with Centos, or are there
> recommended ways to tune it for better performance?

LVM snapshots are a serious performance drain on all systems, even
Microsoft's VSS service puts a heavy heavy strain on IO since both
do COW.

I get around that at work here by having a second cheapo NAS box
that I replicate my files to and do snapshots there and serve them
up there as a separate snapshot server, man that machine's IO gets
punished!

I'd imagine if you had sufficient memory that you could set up a
large RAM-disk, make a PV out of it and add it to your VG, make sure
regular LVs do not get created on it but have your snapshots there.

Snapshots would not persist between reboots, but for temporary
snapshots it sure beats the alternative. Of course you need to make
sure your VG can recover from the loss of this PV though.

> Thanks for any input!
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
> 

______________________________________________________________________
This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.