On Fri, January 23, 2015 12:54 pm, Patrick Flaherty wrote: >> >> I do have two centos 6.6 servers. With a "performance optimized" rsync I >> get an speed of 15 - 20 MB/s >> >> The options I use are: >> >> rsync -aHAXxv --numeric-ids --progress -e "ssh -T -c arcfour -o >> Compression=no -x" >> >> If I copy files by smb to/from the servers I do get 60 - 80 MB/s, a dd >> (r/w) on the storages attached gives 90 MB/s on the 1Gbit ISCSI (Source >> Server) and up to 600MB/s on the 10Gbit ISCSI (Destination Server) >> storage. >> >> Both servers have plenty of memory and cpu usage looks low. >> >> Currently we dont use jumbo frames. Network over all usage is moderate >> to low. There are no special sysctl tweeks yet in use. >> >> As mentioned, I'm confused that even with SMB I do get 3 to 4 times >> better performance. >> >> Any hint and suggestion to track that problem down is welcome! >> > > Not an expert in rsync/ssh, but I'm pretty sure it's ssh's tcp window > size > that is the slowness. ssh is trying to leak the minimal amount of > information to anyone eavesdropping. If speed is your main concern, > http://psc.edu/index.php/hpn-ssh, or rsync to an nfs mount. I'm not certain what the problem could be. But enabling jumbo packets would be the fist thing I would try, after turning off firewall to test if it isn't involved. Joining "other advises" move: we usually use bbftp and gridftp for massive data transfers. Bbftp: http://doc.in2p3.fr/bbftp/ gridftp is available with globus installation. Just my $0.02 Valeri ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++