On Tue, May 1, 2007 12:34 am, Jamie Lists wrote:

The short version. Copying over SFTP to our centos boxes maxes out at 2MB a sec.

The Question: Is there some sort of speed limitation somewhere in the ssh/sftp daemon?

Does anyone have any idea why this could be happening?

The long version We have centos 4.6 installed on 2 different machines, different hardware, different nics. Even on 2 different subnets.

I've tried different switches, 1000 Full/Half 100 Full/Half and still get the same speed. It maxes out at 2MB a sec.

Same switch, network, client using sftp to our xserves gets 10MB a sec. Basically filling the pipe of the network.

The only thing that is the same on both the linux boxes is the distro. So i thought maybe someone may have come across this before.

I haven't tried other protocols yet (FTP and HTTP) I'll check that tomorrow.

Thanks, Jamie

I did some quick research on google a few min ago. Thinking back I have never seen higher than 2MB/s transfer speed using sftp between a Windows and Linux machine. I have however seen much higher transfer speeds using sftp/scp between 2 linux machines.

A few people mentioned they believe the slowness is with the implementation of sftp/scp/ssh in the Windows app.

Try doing an sftp or scp transfer between the 2 CentOS machines and look at the transfer rate. You could also try doing a transfer using Cygwin or maybe some other win32 ports. However it may just be somehting strange with Windows.

...While typing this I figured I would do some additional testing. I downloaded VanDyke SecureFX (VanDyke has some of the best SSH implementations I have seen for windows). My transfer rate was approx 10MB/s on a 100Mbit connection. Whereas with WinSCP and FileZilla it was only approx 2MB/s

On Apr 30, 2007, at 9:34 PM, Jamie Lists wrote:

The Question: Is there some sort of speed limitation somewhere in the ssh/sftp daemon?

Here's a couple potential problems

1. CPU usage. sftp is more intensive than CPU. Don't run out the Mac even though it worked fine with another machine - the algorithm negotiation might have gone differently. Look at the CPU utilization with a tool like top. The pessimistic assumption is that no bytes are being sent during the percentage of the time the CPU is busy.

2. TCP windows. This one's a bit complex to expain:

The deal is that between sftp on machine A sending bytes and sftp on machine B receiving them, machine A's kernel is responsible for retrying sends until machine B's kernel acknowledges them (thus it needs to keep them in a send buffer), and machine B's kernel is responsible for buffering them until the sftp process is ready to deal with them.

Machine B advertises how many bytes are free in its receive buffer through the TCP window. This number is important for performance: that's the number of bytes machine B can send before receiving an acknowledgement, so if it's less than (ping time) * (bandwidth), it can limit throughput.

So first, what is the ping time between these two machines? Calculate this number accordingly, which is usually called the bandwidth*delay product.

Now run sftp between the machines while running tcpdump. On your Macintosh, try something like this:

$ sudo /usr/sbin/tcpdump -ni en0 'port ssh and tcp-syn != 0'

That will dump out the initiating packets of the connection only. For me, an ssh connection from my OS X 10.4.9 machine to my CentOS 5 machine looks like this:

18:09:48.824252 IP 10.4.253.189.53581 > 10.4.254.87.22: S 1900985803:1900985803(0) win 65535 <mss 1460,nop,wscale 0,nop,nop,timestamp 468743802 0,sackOK,eol> 18:09:48.824434 IP 10.4.254.87.22 > 10.4.253.189.53581: S 4196469947:4196469947(0) ack 1900985804 win 5792 <mss 1460,sackOK,timestamp 280331387 468743802,nop,wscale 7>

My Macintosh (10.4.253.189) is advertising a TCP window size of 65535 bytes.

My Linux box (10.4.254.87) is doing something a bit more complicated. When TCP was originally created, they didn't foresee the need for window sizes greater than 65535. It's using an option called TCP window scaling to multiply its advertised window size by 2^7. So it's advertising a window size of 741k...if the Mac supports window scaling as well...otherwise, a dismal 5792 bytes. In this case, my Mac does claim support for window scaling, so the full size is used.

I'm using 100 Mbps Ethernet and my round-trip ping times between the machines are at most 0.3 ms, so full utilization requires this window size:

100 * 10^6 bits/sec * (byte / 8 bits) * (0.3 * 10^-3 s) = 3000 bytes.

so both window sizes are plenty. But if I were using gigabit Ethernet with a full millisecond of latency:

100 * 10^9 bits/sec * (byte / 8 bits) * (10^-3 s) = 1000000 bytes = 100 kbytes

I could send at full speed from the Mac to the Linux machine but not the other way around, as the Mac's receive window is too small.