On Fri, January 23, 2015 12:54 pm, Patrick Flaherty wrote:
I do have two centos 6.6 servers. With a "performance optimized" rsync I get an speed of 15 - 20 MB/s
The options I use are:
rsync -aHAXxv --numeric-ids --progress -e "ssh -T -c arcfour -o Compression=no -x"
If I copy files by smb to/from the servers I do get 60 - 80 MB/s, a dd (r/w) on the storages attached gives 90 MB/s on the 1Gbit ISCSI (Source Server) and up to 600MB/s on the 10Gbit ISCSI (Destination Server) storage.
Both servers have plenty of memory and cpu usage looks low.
Currently we dont use jumbo frames. Network over all usage is moderate to low. There are no special sysctl tweeks yet in use.
As mentioned, I'm confused that even with SMB I do get 3 to 4 times better performance.
Any hint and suggestion to track that problem down is welcome!
Not an expert in rsync/ssh, but I'm pretty sure it's ssh's tcp window size that is the slowness. ssh is trying to leak the minimal amount of information to anyone eavesdropping. If speed is your main concern, http://psc.edu/index.php/hpn-ssh, or rsync to an nfs mount.
I'm not certain what the problem could be. But enabling jumbo packets would be the fist thing I would try, after turning off firewall to test if it isn't involved. Joining "other advises" move: we usually use bbftp and gridftp for massive data transfers. Bbftp:
gridftp is available with globus installation.
Just my $0.02
Valeri
++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++