On 07/26/2014 07:04 AM, Jerry Franz wrote: > On 07/25/2014 03:33 PM, Benjamin Smith wrote: >> takes between 1 and 2 days, system load depending. We had to give up >> on rsync for backups in this context a while ago - we just couldn't >> get a "daily" backup more often then about 2x per week. Now we're >> using ZFS + send/receive to get daily backup times down into the "sub >> 60 minutes" range, and I'm just going to bite the bullet and >> synchronize everything at the application level over the next week. >> Was just looking for a shortcut... > Here is an evil thought. Is this possible for you do? > > 1) Setup a method to obtain a RW lock for updates on the original filesystem > > 2) Use rsync to create a gross copy of the original (yes, it will be > slightly out of phase, > but stick with me for a bit) on the new filesystem on top of LVM2 > on top of a RAID1 > volume to make the next step much more efficient. > > 3) Perform the following loop: > a) Set the updates lock on original filesystem > b) rsync a *subset* sub-directory of the original filesystem such > that you can complete it in, at worst, only a second or two > c) Rename the original directory to some safe alternative (safety > first)... > d) Put a symlink in place of the original directory pointing to the > newly synced file system sub-directory > e) Release the mutex lock > f) Repeat a-e until done > > 4) Switch over operations to the new filesystem That's essentially what we do to re-sync our production file stores. Once I move to ZFS, though it won't be an issue.