[CentOS] How to speed up Rsync transfers

[Dan Carl]

----- Original Message ----- 
From: "William L. Maltby" <CentOS4Bill at triad.rr.com>
To: "CentOS General List" <centos at centos.org>
Sent: Saturday, February 23, 2008 12:32 PM
Subject: Re: [CentOS] How to speed up Rsync transfers


> On Sat, 2008-02-23 at 10:38 -0600, Dan Carl wrote:
>> ----- Original Message ----- 
>> From: "nate" <centos at linuxpowered.net>
>> To: <centos at centos.org>
>> Sent: Friday, February 22, 2008 10:33 PM
>> Subject: RE: [CentOS] How to speed up Rsync transfers
>>
>>
>> > Dan Carl wrote:
>> ><snip>
>
>> > - Do you know what sort of bandwidth your "supposed" to have from your
>> >  ISP?
>> source server business DSL 1.5m down / 878k up
>> distination server T1 colo at a large ISP.
>> >
>> > nate
>> Sounds like I'll be stuck with the tranfer rate I'm getting.
>
> In that case, it sounds like you need a local staging that can be
> quickly done before starting upload sync. Then the upload can run 24/7.
> How you might want to deal with new updates that happen before the
> previous upload finishes is going to be an interesting problem.
>
This is exactly the situation I'm trying to avoid.
Right now its less than 2GB new/edited images a day so the rsync backup 
finishes before the script runs again.
But I can't take it for granted that this will always be the case.
Any ideas would be appreciated. What do you mean by local staging?
I'd like the backup to run from 7pm to 7am and then if it didn't finish to 
resume again the next night.
That way when nothing was added/edited on the weekends the backup can catch 
up.
Dan