On Fri, Jul 25, 2014 at 3:08 PM, Benjamin Smith <lists at benjamindsmith.com> wrote: > On 07/25/2014 12:12 PM, Michael Hennebry wrote: >> Is there soome reason that the existing files cannot >> be accessed while they are being copied to the raid? >> > Sheer volume. With something in the range of 100,000,000 small files, it > takes a good day or two to rsync. This means that getting a consistent > image without significant downtime is impossible. I can handle a few > minutes, maybe an hour. Much more than that and I have to explore other > options. (In this case, it looks like we'll be biting the bullet and > switching to ZFS) Rsync is really pretty good at that, especially the 3.x versions. If you've just done a live rsync (or a few so there won't be much time for changes during the last live run), the final one with the system idle shouldn't take much more time than a 'find' traversing the same tree. If you have space and time to test, I'd time the third pass or so before deciding it won't work (unless even find would take too long).