[CentOS] Sftp slow on both our centos installs

Scott Lamb slamb at slamb.org
Wed May 2 01:42:06 UTC 2007


On Apr 30, 2007, at 9:34 PM, Jamie Lists wrote:

> The Question:
> Is there some sort of speed limitation somewhere in the ssh/sftp  
> daemon?

Here's a couple potential problems

1. CPU usage. sftp is more intensive than CPU. Don't run out the Mac  
even though it worked fine with another machine - the algorithm  
negotiation might have gone differently. Look at the CPU utilization  
with a tool like top. The pessimistic assumption is that no bytes are  
being sent during the percentage of the time the CPU is busy.

2. TCP windows. This one's a bit complex to expain:

The deal is that between sftp on machine A sending bytes and sftp on  
machine B receiving them, machine A's kernel is responsible for  
retrying sends until machine B's kernel acknowledges them (thus it  
needs to keep them in a send buffer), and machine B's kernel is  
responsible for buffering them until the sftp process is ready to  
deal with them.

Machine B advertises how many bytes are free in its receive buffer  
through the TCP window. This number is important for performance:  
that's the number of bytes machine B can send before receiving an  
acknowledgement, so if it's less than (ping time) * (bandwidth), it  
can limit throughput.

So first, what is the ping time between these two machines? Calculate  
this number accordingly, which is usually called the bandwidth*delay  
product.

Now run sftp between the machines while running tcpdump. On your  
Macintosh, try something like this:

     $ sudo /usr/sbin/tcpdump -ni en0 'port ssh and tcp-syn != 0'

That will dump out the initiating packets of the connection only. For  
me, an ssh connection from my OS X 10.4.9 machine to my CentOS 5  
machine looks like this:

     18:09:48.824252 IP 10.4.253.189.53581 > 10.4.254.87.22: S  
1900985803:1900985803(0) win 65535 <mss 1460,nop,wscale  
0,nop,nop,timestamp 468743802 0,sackOK,eol>
     18:09:48.824434 IP 10.4.254.87.22 > 10.4.253.189.53581: S  
4196469947:4196469947(0) ack 1900985804 win 5792 <mss  
1460,sackOK,timestamp 280331387 468743802,nop,wscale 7>

My Macintosh (10.4.253.189) is advertising a TCP window size of 65535  
bytes.

My Linux box (10.4.254.87) is doing something a bit more complicated.  
When TCP was originally created, they didn't foresee the need for  
window sizes greater than 65535. It's using an option called TCP  
window scaling to multiply its advertised window size by 2^7. So it's  
advertising a window size of 741k...if the Mac supports window  
scaling as well...otherwise, a dismal 5792 bytes. In this case, my  
Mac does claim support for window scaling, so the full size is used.

I'm using 100 Mbps Ethernet and my round-trip ping times between the  
machines are at most 0.3 ms, so full utilization requires this window  
size:

     100 * 10^6 bits/sec * (byte / 8 bits) * (0.3 * 10^-3 s) = 3000  
bytes.

so both window sizes are plenty. But if I were using gigabit Ethernet  
with a full millisecond of latency:

     100 * 10^9 bits/sec * (byte / 8 bits) * (10^-3 s) = 1000000  
bytes = 100 kbytes

I could send at full speed from the Mac to the Linux machine but not  
the other way around, as the Mac's receive window is too small.

-- 
Scott Lamb <http://www.slamb.org/>





More information about the CentOS mailing list