[CentOS] network copy performance is poor (rsync) - debugging suggestions?

[Valeri Galtsev]

On Fri, January 23, 2015 12:54 pm, Patrick Flaherty wrote:
>>
>> I do have two centos 6.6 servers. With a "performance optimized" rsync I
>> get an speed of 15 - 20 MB/s
>>
>> The options I use are:
>>
>> rsync -aHAXxv --numeric-ids --progress -e "ssh -T -c arcfour -o
>> Compression=no -x"
>>
>> If I copy files by smb to/from the servers I do get 60 - 80 MB/s, a dd
>> (r/w) on the storages attached gives 90 MB/s on the 1Gbit ISCSI (Source
>> Server) and up to 600MB/s on the 10Gbit ISCSI (Destination Server)
>> storage.
>>
>> Both servers have plenty of memory and cpu usage looks low.
>>
>> Currently we dont use jumbo frames. Network over all usage is moderate
>> to low. There are no special sysctl tweeks yet in use.
>>
>> As mentioned, I'm confused that even with SMB I do get 3 to 4 times
>> better performance.
>>
>> Any hint and suggestion to track that problem down is welcome!
>>
>
>  Not an expert in rsync/ssh, but I'm pretty sure it's ssh's tcp window
> size
> that is the slowness. ssh is trying to leak the minimal amount of
> information to anyone eavesdropping. If speed is your main concern,
> http://psc.edu/index.php/hpn-ssh, or rsync to an nfs mount.

I'm not certain what the problem could be. But enabling jumbo packets
would be the fist thing I would try, after turning off firewall to test if
it isn't involved. Joining "other advises" move: we usually use bbftp and
gridftp for massive data transfers. Bbftp:

http://doc.in2p3.fr/bbftp/

gridftp is available with globus installation.

Just my $0.02

Valeri

++++++++++++++++++++++++++++++++++++++++
Valeri Galtsev
Sr System Administrator
Department of Astronomy and Astrophysics
Kavli Institute for Cosmological Physics
University of Chicago
Phone: 773-702-4247
++++++++++++++++++++++++++++++++++++++++