On 04.12.2013 14:05, nux at li.nux.ro wrote: >>> >>On 04.12.2013 14:05, John Doe wrote: >>>>> >>>>From: Lists<lists at benjamindsmith.com> >>>>> >>>> >>>>>>> >>>>>>Our next big test is to try out ZFS filesystem send/receive in >>>>>>> >>>>>>lieu >>>>>>> >>>>>>of >>>>>>> >>>>>>our current backup processes based on rsync. Rsync is a fabulous >>>>>>> >>>>>>tool, >>>>>>> >>>>>>but is beginning to show performance/scalability issues dealing >>>>>>> >>>>>>with >>>>>>> >>>>>>the >>>>>>> >>>>>>many millions of files being backed up, and we're hoping that ZFS >>>>>>> >>>>>>filesystem replication solves this. >>>>> >>>> >>>>> >>>>Not sure if I already mentioned it but maybe have a look at: >>>>> >>>>?http://code.google.com/p/lsyncd/ >>> >>I'm not so sure inotify works well with millions of files, not to >>> >>mention it uses rsync. :D >>> >> >>> >>-- Sent from the Delta quadrant using Borg technology! Nux! >> > >> >I can attest to the usefulness of 'lsyncd' for large numbers of files >> >(our file server has almost 2 million in active use, with a second >> >backup server that's lsync'd to the first. >> > >> >Things to note: >> >- Yes, lsyncd does use rsync, but it issues an 'exclude *' followed by >> >the list of only the file(s) that need updating at that moment. >> > >> >- The inotify service can be jacked waaaay up (three kernel >> >parameters) >> >to handle millions of files if you wish. Just make sure you have lots >> >of RAM. > Be careful with it. Sadly I found out that inotify would consistently > fail on InnoDB files (ibd); I had to use stupid while loops and check > mtimes to perform some stuff that inotify-cron would've done much more > elegantly ... > > -- Sent from the Delta quadrant using Borg technology! Nux! Interesting point, something I didn't know. Fortunately in my case there are no db files involved directly, just db dumps wrapped in a tarball along with other associated stuff, sent from other servers. I would expect that lsync'ing db files could be a nasty non-stop process if the database is constantly being updated, so using db tools for replication would be best, configuring inotify/lsyncd to ignore the db directories. I believe by default that lsyncd instructs rsync to do whole-file transfers, so a large db could be a real problem. Thanks for the important heads-up!