[CentOS] ZFS on Linux testing

Wed Dec 18 01:59:03 UTC 2013
Lists <lists at benjamindsmith.com>

On 12/14/2013 08:50 AM, Chuck Munro wrote:
> Hi Ben,
>
> Yes, the initial replication of a large filesystem is *very* time
> consuming!  But it makes sleeping at night much easier.  I did have to
> crank up the inotify kernel parameters by a significant amount.
>
> I did the initial replication using rsync directly, rather than asking
> lsyncd to do it.  I notice that if I reboot the primary server, it takes
> a while for the inotify tables to be rebuilt ... after that it's smooth
> sailing.

I may be being presumptuous, and if so, I apologize in advance...

It sounds to me like you might consider a disk-to-disk backup solution. 
I could suggest dirvish, BackupPC, or our own home-rolled rsync-based 
solution that works rather well: http://www.effortlessis.com/backupbuddy/

Note that with these solutions you get multiple save points that are 
deduplicated with hardlinks so you can (usually) keep dozens of save 
points in perhaps 2x the disk space of a single copy. Also, because of 
this, you can go back a few days / weeks / whatever when somebody 
deletes a file. In our case, we make the backed up directories available 
via read-only ftp so that end users can recover their files.

I don't know if dirvish offers this, but backupbuddy also allows you to 
run pre and post backup shell scripts, which we use (for example) for 
off-site archiving to permanent storage since backup save points expire.

-Ben

> If you want to prevent deletion of files from your replicated filesystem
> (which I do), you can modify the rsync{} array in the lsyncd.lua file by
> adding the line 'delete = false' to it.  This has saved my butt a few
> times when a user has accidentally deleted a file on the primary server.
>
> I agree that filesystem replication isn't really a backup, but for now
> it's all I have available, but at least the replicated fs is on a
> separate machine.
>
> As a side note for anyone using a file server for hosting OS-X Time
> Machine backups, the 'delete' parameter in rsync{} must be set to 'true'
> in order to prevent chaos should a user need to point their Mac at the
> replicate filesystem (which should be a very rare event).  I put all TM
> backups in a separate ZFS sub-pool for this reason.
>