[CentOS] recommendations for copying large filesystems

[Dag Wieers]

On Sun, 22 Jun 2008, Raja Subramanian wrote:

> On Sun, Jun 22, 2008 at 3:36 AM, Rainer Duffner <rainer at ultra-secure.de> wrote:
>> Now that I know the details - I don' think this is going to work. Not with
>> 100 TB of data. It kind-of-works with 1 TB.
>> Can anybody comment on the feasibility of rsync on 1 million files?
>
> rsync always broke on my filesystems with 200-300k files
> due to out of memory errors (my box had 2GB RAM).

I have done 700k and 800k files transfers (including hardlinks), but 
indeed it could take a while to compute the transferlist. Newer rsync 
versions bring down the amount of memory needed drastically. That is one 
of the reasons I offer a recent rsync in RPMforge. There is almost never a 
good reason to use a dated rsync.

-- 
--   dag wieers,  dag at centos.org,  http://dag.wieers.com/   --
[Any errors in spelling, tact or fact are transmission errors]