[CentOS] Best practices for copying lots of files machine-to-machine

Wed May 17 16:14:12 UTC 2017
Nux! <nux at li.nux.ro>


Rsync can maintain symlinks, hardlinks and give you a progress report as well; not to mention it can resume interruptions should they occur.

Having said that, even with your space problem, it is possible to use tar to pack files during transfer, on the fly which should be faster than rsync.
Just search for "tar over ssh".

Sent from the Delta quadrant using Borg technology!


----- Original Message -----
> From: "ken" <gebser at mousecar.com>
> To: "CentOS mailing list" <centos at centos.org>
> Sent: Wednesday, 17 May, 2017 17:03:13
> Subject: [CentOS] Best practices for copying lots of files machine-to-machine

> An entire filesystem (~180g) needs to be copied from one local linux
> machine to another.  Since both systems are on the same local subnet,
> there's no need for encryption.
> I've done this sort of thing before a few times in the past in different
> ways, but wanted to get input from others on what's worked best for them.
> One consideration is that the source filesystem contains quite a few
> hardlinks and symlinks and of course I want to preserve these, and
> preserve all timestamps and ownerships and permissions as well.
> Maintaining the integrity of this metadata and the integrity of the
> files themselves if of course the top priority.
> Speed is also a consideration, but having done this before, I find it
> even more important to have a running progress report or log so I can
> see how the session is proceeding and approximately how much longer it
> will be until finished... and too to see if something's hung up.
> One other consideration:  There isn't much disk space left on the source
> machine, so creating a tar file, even compressed, isn't an option.
> What relevant methods have you been impressed by?
> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> https://lists.centos.org/mailman/listinfo/centos