I do have two centos 6.6 servers. With a "performance optimized" rsync I get an speed of 15 - 20 MB/s
The options I use are:
rsync -aHAXxv --numeric-ids --progress -e "ssh -T -c arcfour -o Compression=no -x"
If I copy files by smb to/from the servers I do get 60 - 80 MB/s, a dd (r/w) on the storages attached gives 90 MB/s on the 1Gbit ISCSI (Source Server) and up to 600MB/s on the 10Gbit ISCSI (Destination Server) storage.
Both servers have plenty of memory and cpu usage looks low.
Currently we dont use jumbo frames. Network over all usage is moderate to low. There are no special sysctl tweeks yet in use.
As mentioned, I'm confused that even with SMB I do get 3 to 4 times better performance.
Any hint and suggestion to track that problem down is welcome!
Not an expert in rsync/ssh, but I'm pretty sure it's ssh's tcp window size that is the slowness. ssh is trying to leak the minimal amount of information to anyone eavesdropping. If speed is your main concern, http://psc.edu/index.php/hpn-ssh, or rsync to an nfs mount.