[CentOS] Best practices for copying lots of files machine-to-machine
Robert Moskowitz
rgm at htt-consult.com
Wed May 17 16:16:23 UTC 2017
On 05/17/2017 12:03 PM, ken wrote:
> An entire filesystem (~180g) needs to be copied from one local linux
> machine to another. Since both systems are on the same local subnet,
> there's no need for encryption.
>
> I've done this sort of thing before a few times in the past in
> different ways, but wanted to get input from others on what's worked
> best for them.
>
> One consideration is that the source filesystem contains quite a few
> hardlinks and symlinks and of course I want to preserve these, and
> preserve all timestamps and ownerships and permissions as well.
> Maintaining the integrity of this metadata and the integrity of the
> files themselves if of course the top priority.
>
> Speed is also a consideration, but having done this before, I find it
> even more important to have a running progress report or log so I can
> see how the session is proceeding and approximately how much longer it
> will be until finished... and too to see if something's hung up.
>
> One other consideration: There isn't much disk space left on the
> source machine, so creating a tar file, even compressed, isn't an option.
>
> What relevant methods have you been impressed by?
I use rsync for such work. It is good at maintaining hard and sym links
and timestamps. It can give you a running progress as well.
One thing I have learned is that crud happens and I loose my local
session for some stupid reason or another, thus I often run rsync in a
screen shell that I can easily reconnect to.
More information about the CentOS
mailing list