On Dec 3, 2010, at 2:33 PM, Gordon Messmer <yinyang at eburg.com> wrote: > On 12/02/2010 04:28 AM, Peter Kjellström wrote: >> IMO lots of people waste time on jumbo frames when there's really no (or >> very little) need. > > That depends on the protocols in use and your TCP window configuration. > Streaming protocols like HTTP may benefit less from jumbo frames > (except, as has been noted, for reduced CPU use) in some configurations, > but latency-sensitive protocols like CIFS and NFS will benefit greatly. If the protocol is latency sensitive then jumbo frames are BAD as it adds more latency because frames take longer to fill, longer to transmit and thus other conversations have to wait longer (poor pipelining/interlacing). CIFS/NFS aren't really latency sensitive protocols though. If a protocol has a big TCP window then it will not tend to be latency sensitive. A protocol that is latency sensitive would be iSCSI with sequential IO. Each iSCSI PDU is (default) 8K which is a lot smaller then say the 32K CIFS/NFS protocol units one sees, so bumping up the latency just a tad will mean you will only get 25MB/s 4k sequential throughput instead of 40MB/s. Bumping up the iSCSI PDU helps but limits the total number of simultaneous iSCSI PDUs a target can handle. That's why typically on 1Gbe iSCSI, it's better to use 1500 MTU instead of 9000. With 10Gbe though it uses a lot of CPU to send 1500 byte frames, so there it's better using 9000 MTU. Everything is a compromise due to finite resources. -Ross