[CentOS] rsync question: building list taking forever

Mon Oct 20 02:05:42 UTC 2014
Tim Dunphy <bluethundr at gmail.com>

>
> > Don't forget that the time taken to build the file list is a function of
> > the number of files present, and not their size. If you have many
> millions
> > of small files, it will indeed take a very long time. Over sshfs with
> > a slowish link, it could be days.
> >
> > ....and it may end up failing silently or noisily anyway.


Ahhh, but isn't that part of the beauty of adventure that being a linux
admin is all about? *twitch*

On Sun, Oct 19, 2014 at 8:27 PM, Cliff Pratt <enkiduonthenet at gmail.com>
wrote:

> On Mon, Oct 20, 2014 at 7:57 AM, Steve Thompson <smt at vgersoft.com> wrote:
>
> > On Sun, 19 Oct 2014, Keith Keller wrote:
> >
> >  I suspect that sshfs's relatively poor performance is having an impact
> >> on your transfer.  I have a 30TB filesystem which I rsync over an
> >> OpenVPN link, and building the file list doesn't take that long (maybe
> >> an hour?).  (The links themselves are reasonably fast; if yours are not
> >> that would have a negative impact too.)
> >>
> >
> > Don't forget that the time taken to build the file list is a function of
> > the number of files present, and not their size. If you have many
> millions
> > of small files, it will indeed take a very long time. Over sshfs with
> > a slowish link, it could be days.
> >
> > ....and it may end up failing silently or noisily anyway.
>
> Cheers,
>
> Cliff
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>



-- 
GPG me!!

gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B