-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of t m Sent: Tuesday, March 13, 2007 7:41 PM To: CentOS mailing list Subject: [CentOS] LVM performance with snapshots
I've been experimenting with LVM and snapshots under Centos 4.4 recently, and have found performance to be horrible while a snapshot of a logical volume exists.
I expected some performance degradation with a snapshot in place due to my understanding of the copy-on-write behavior, but my system becomes almost unusable when writing to my system with a snapshot in place. For example, I can write a 4 gigabyte file in about 16 seconds
What speed do you get if you mount the file system with the 'sync' option?
4GB in 16 seconds equates to 268MB/s, and at 4k block size you either have a phenomenal fiber channel based RAID system or a whole lot of memory that provides a huge write-back cache, or both.
with no snapshots in place, but the same file either takes about 4 minutes to write, or my systems seems to hang when a snapshot exists.
Yes, now the LVM copy-on-write is kicking in and for each block written it now needs to read the existing block, seek to the snapshot, write the existing block, seek back to position and write the new block.
The fact that the penalty is so high, and the iowait times are high lets me think that you do not have that massive fiber channel backed RAID setup.
The top and iostat commands seem to show very high iowait times and very low actual I/O throuhput, and pdflush seems to be quite busy doing something. Removing the snapshot immediately brought performance back to normal.
pdflush is writing out write-cache to disk. The disks are taking so long to service the requests now that the write cache is backing up.
Are LVM snapshots not recommended with Centos, or are there recommended ways to tune it for better performance?
LVM snapshots are a serious performance drain on all systems, even Microsoft's VSS service puts a heavy heavy strain on IO since both do COW.
I get around that at work here by having a second cheapo NAS box that I replicate my files to and do snapshots there and serve them up there as a separate snapshot server, man that machine's IO gets punished!
I'd imagine if you had sufficient memory that you could set up a large RAM-disk, make a PV out of it and add it to your VG, make sure regular LVs do not get created on it but have your snapshots there.
Snapshots would not persist between reboots, but for temporary snapshots it sure beats the alternative. Of course you need to make sure your VG can recover from the loss of this PV though.
Thanks for any input! _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.