[CentOS] ZFS on Linux testing effort

[Nux!]

On 05.12.2013 22:46, Chuck Munro wrote:
>> On 04.12.2013 14:05, John Doe wrote:
>>>> From: Lists<lists at benjamindsmith.com>
>>>> 
>>>>>> Our next big test is to try out ZFS filesystem send/receive in 
>>>>>> lieu
>>>>>> of
>>>>>> our current backup processes based on rsync. Rsync is a fabulous
>>>>>> tool,
>>>>>> but is beginning to show performance/scalability issues dealing 
>>>>>> with
>>>>>> the
>>>>>> many millions of files being backed up, and we're hoping that ZFS
>>>>>> filesystem replication solves this.
>>>> 
>>>> Not sure if I already mentioned it but maybe have a look at:
>>>> ?http://code.google.com/p/lsyncd/
>> I'm not so sure inotify works well with millions of files, not to
>> mention it uses rsync. :D
>> 
>> -- Sent from the Delta quadrant using Borg technology! Nux!
> 
> I can attest to the usefulness of 'lsyncd' for large numbers of files
> (our file server has almost 2 million in active use, with a second
> backup server that's lsync'd to the first.
> 
> Things to note:
> - Yes, lsyncd does use rsync, but it issues an 'exclude *' followed by
> the list of only the file(s) that need updating at that moment.
> 
> - The inotify service can be jacked waaaay up (three kernel 
> parameters)
> to handle millions of files if you wish.  Just make sure you have lots
> of RAM.

Be careful with it. Sadly I found out that inotify would consistently 
fail on InnoDB files (ibd); I had to use stupid while loops and check 
mtimes to perform some stuff that inotify-cron would've done much more 
elegantly ...

-- 
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro