[CentOS] How to speed up Rsync transfers

[Dan Carl]

> -----Original Message-----
> From: centos-bounces at centos.org [mailto:centos-bounces at centos.org]On
> Behalf Of nate
> Sent: Friday, February 22, 2008 11:06 AM
> To: centos at centos.org
> Subject: Re: [CentOS] How to speed up Rsync transfers
>
>
> Dan Carl wrote:
> > Is there a way to speed up rsync transfers?
> > I tested the bandwidth with iperf (recommended to me in an
> earlier post &
> > worked well)its as advertised by my ISP's around 740KB/sec.
> > When I manually run my rsync script with the --progress switch
> the transfers
> > are around 100KB/sec.
> > I googled this and the only thing I found had to do with the TCP window
> > which I understand to be the limiting factor. But if this is
> true how can I
> > ftp stuff at 300KB/sec? (someone please enlighten me)
> >
> > I'm backing up jpg files and some days they add 5+GB of images.
> > My goal was to backup the images nightly, but at the 100KB/sec
> rate that's
> > not possible.
>
> What options are you using with rsync ? Stay away from the  -z option
> if your copying jpegs. That'll slow things down quite a bit. I have
> no problem using rsync to copy at 20+MBytes/second with the default
> tcp window size on gigabit networks(using rsync over SSH). And I
> can achieve ~700kByte/s on a 10Mbit link over a VPN between two sites(
> about 40 miles apart), again, default tcp settings across the board.
>
> Also note that some ISPs, such as Comcast have features built into their
> services that provide an initial "power boost" for a few seconds to
> speed up file transfers, then are quickly throttled down to "normal"
> levels. In this case your bandwidth test with the ISP may not of lasted
> long enough to show your "true" long, sustained available bandwidth.
>
> The options I use:
>
> rsync -ave ssh (local file) remote_server:(remote file)
>
> or rsync -ave ssh --progress (rest of command)
>
> nate

I don't do anything that special
rsync -ae 'ssh -p 2112' --progress $SOURCEPATH $DESTUSER@$DESTHOST:$DESTPATH
or
rsync -a --progress --rsh='ssh -p 2112' $SOURCEPATH
$DESTUSER@$DESTHOST:$DESTPATH

no real speed difference etiher way.

I just ran a test from one local box the another on a 100Mbit link and the
fastest transfer OI got was 10MBytes/second but most were between
1MBytes/second and 4MBytes/second.

Dan

> _______________________________________________
> CentOS mailing list
> CentOS at centos.org
> http://lists.centos.org/mailman/listinfo/centos
>