[CentOS] rsync question

Tue Jul 31 23:24:10 UTC 2012
Smithies, Russell <Russell.Smithies at agresearch.co.nz>

As far as I can see timestamp and size is the default.
I've turned off compression and I think I'm getting better throughput.
Running 4 rsync tasks and getting sustained transfers for several hours of just over 800Mb/sec :- )


-----Original Message-----
From: centos-bounces at centos.org [mailto:centos-bounces at centos.org] On Behalf Of John R Pierce
Sent: Tuesday, 31 July 2012 5:16 p.m.
To: centos at centos.org
Subject: Re: [CentOS] rsync question

On 07/30/12 10:05 PM, Smithies, Russell wrote:
> I'm trying to rsync a 8TB data folder containing squillions of small files and it's taking forever (i.e. weeks) to get anywhere.
> I'm assuming the slow bit is check-summing everything with a single 
> CPU (even though it's on a 12-core server ;-(  ) Is it possible to do something simple like scp the whole dir in one go so they're duplicates in the first instance, then get rsync to just keep them in sync without an initial transfer?

use the rsync mode that goes off file timestamp and size.  the checksuming block algorithm is only useful on large files that get small random block changes.

john r pierce                            N 37, W 122
santa cruz ca                         mid-left coast

CentOS mailing list
CentOS at centos.org
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.