[CentOS] recommendations for copying large filesystems
dag at centos.org
Sun Jun 22 20:32:39 UTC 2008
On Sun, 22 Jun 2008, Raja Subramanian wrote:
> On Sun, Jun 22, 2008 at 3:36 AM, Rainer Duffner <rainer at ultra-secure.de> wrote:
>> Now that I know the details - I don' think this is going to work. Not with
>> 100 TB of data. It kind-of-works with 1 TB.
>> Can anybody comment on the feasibility of rsync on 1 million files?
> rsync always broke on my filesystems with 200-300k files
> due to out of memory errors (my box had 2GB RAM).
I have done 700k and 800k files transfers (including hardlinks), but
indeed it could take a while to compute the transferlist. Newer rsync
versions bring down the amount of memory needed drastically. That is one
of the reasons I offer a recent rsync in RPMforge. There is almost never a
good reason to use a dated rsync.
-- dag wieers, dag at centos.org, http://dag.wieers.com/ --
[Any errors in spelling, tact or fact are transmission errors]
More information about the CentOS