On Friday, December 24, 2010 06:40:06 am Ryan Wagoner wrote:

LVM is just like the name implies a logical volume manager. It allows you to easily combine and carve space from physical disks. It doesn't provide any redundancy. If you want redundancy you either need to use the LVM mirror capability, linux software raid with mdadm, or a hardware raid card.

It's been said before, and never seems to be said often enough: backup your data!

IMHO, very few people really need RAID. In many (most?) cases, the added complexity of RAID is as likely to cause an increase of failure rate similar to or greater than the reduction of failure rate caused by the resiliancy of hardware. RAID won't protect you if you issue a perfectly legitimate command to delete data that was made in error. I once thought I needed RAID, and since realized the error in my ways, finding that the cases where RAID helped (one!) was vastly outnumbered by the cases where it made no difference (10? 11?) or actually worked against me. (2) Now, I don't bother with RAID even though my needs have grown from one server to 16, instead providing redundancy at the machine level: if a server goes, another picks up the load, in most cases automatically, in near-real-time. I can do this because I host a custom-made application with these objectives carefully designed for.

The MUST HAVE option: a backup! Backups provide real, multi-point redundancy, some protection against the "Ooops, I didn't mean to delete that file" scenario, etc. and are there when all else has failed. I'd happily point you to my set of scripts that coordinate disk-to-disk backups over a network with automatically expired snapshot points. http://www.effortlessis.com/thisisnotbackupbuddy.

On Sat, 2010-12-25 at 08:47 -0500, Ryan Wagoner wrote:

On Sat, Dec 25, 2010 at 2:15 AM, Benjamin Smith lists@benjamindsmith.com wrote:

...

IMHO, very few people really need RAID. In many (most?) cases, the added complexity of RAID is as likely to cause an increase of failure rate similar to or greater than the reduction of failure rate caused by the resiliancy of hardware. RAID won't protect you if you issue a perfectly legitimate command to delete data that was made in error. I once thought I needed RAID, and since realized the error in my ways, finding that the cases where RAID helped (one!) was vastly outnumbered by the cases where it made no difference (10? 11?) or actually worked against me. (2) Now, I don't bother with RAID even though my needs have grown from one server to 16, instead providing redundancy at the machine level: if a server goes, another picks up the load, in most cases automatically, in near-real-time. I can do this because I host a custom-made application with these objectives carefully designed for.

I'm not sure why you have so many problems with RAID.

+1

I will never run a production server without RAID. A simple mirror (RAID 1) potentially increases up time and doesn't add much complexity.

+1 And a lowly technician can do it while I'm on vacation.

time. Failing over to another server is always great, but why be one server down for a simple drive failure.

Depending on the application failing to another server is also fraught with issues.

Not to mention the speed increases from RAID 5 or 10.

Speed increase from RAID 10 yes, not RAID 5. http://www.miracleas.com/BAARF/BAARF2.html

On Sat, Dec 25, 2010 at 2:23 PM, Ryan Wagoner rswagoner@gmail.com wrote:

RAID 5 does provide speed increases for read operations. There are still some applications where RAID 5 has its benefits. For a smaller department file server 3-4 TB drives in RAID 5 works great. The money saved can be put towards backups, etc. Having said that I use RAID 10 for most applications.

I've been thinking about the whole backup/redundancy approach to maintaining my home network. Though it is a "home" network, I use it to support my work so though it's not business critical, it can't tolerate much downtime.

Two approaches that I see:

1) Use some form of RAID or mirroring and a backup process to provide recoverability.

2) Maintain a centralized configuration in order to quickly re-build a downed system.

They are not completely separate, but I've been trying to move my recovery philosophy to the latter to minimize costs.

For example, I used to backup my DNS/LDAP server by creating a snapshot of the Xen LVM partition. Recovery was simply a matter of restoring the backup.

The problem with this approach was that the number of virtual machines started to balloon and with it, the storage requirements. Though the images were only 10G to 20G, I had dozens of them. Not to mention that the virtualization hosts keep changing: VMWare Server stopped being free; The upstream vendor moved to KVM instead of Xen.

The approach I'm taking now is to use a combination of Kickstart/Anaconda and cfengine with the goal of removing all host identity from a OS instance.

For my DNS server, for example, I use kickstart to build a standard image with a cfengine client. On bootup, cfengine pulls in the bind configuration and within a few minutes I have a duplicate of the old server. This is also much more resilient to changes in the underlying VM technology.

There are still some kinks to work out. For one, I haven't backed up the cfengine server the same way. Supposedly it's just a matter of making the server its own client, but right now I'm using an image backup and rsync copies to another fileserver.

I'm also trying to integrate a Spacewalk server into the mix. This will allow me to rebuild a system with the exact same packages (right now I just update to the latest). In a real production environment this is critical as some applications may only be certified against particular kernel/glibc/etc. versions.