[CentOS] CentOS 4.4 e1000 and wire-speed

Sat Dec 30 23:37:26 UTC 2006
Ross S. W. Walker <rwalker at medallion.com>

Currently I'm running CentOS 4.4 on a Dell Poweredge 850 with an Intel
Pro/1000 Quad-port adapter.

I seem to be able to only achieve 80% utilization on the adapter, while
on the same box running Fedora Core 5 I was able to reach 99%

I am using iSCSI Enterprise Target as my application and I am using the
nullio feature, it just discards any write and sends back random data
for any read, for my bandwidth test. The IO pattern I use is 1MB block

Is there something in the OS that is throttling the bandwidth as it
seems to be capped at 80%.

I have selinux and iptables disabled, and I have tuned the TCP/IP stack
to mimic the settings under 2.6.17, except bumped up the default IP
send/receiver buffer size for improved UDP transmission over 1Gbps.

The CPU is a P4 Dual Core 3GHz, not top of the line but adequate for my
needs (strictly block io).

Here are the TCP/IP tunables from my sysctl.conf:

# Controls default receive buffer size (bytes)
net.core.rmem_default = 262144

# Controls IP default send buffer size (bytes)
net.core.wmem_default = 262144

# Controls IP maximum receive buffer size (bytes)
net.core.rmem_max = 262144

# Controls IP maximum send buffer size (bytes)
net.core.wmem_max = 262144

# Controls TCP memory utilization (pages)
net.ipv4.tcp_mem = 49152 65536 98304

# Controls TCP sliding receive window buffer (bytes)
net.ipv4.tcp_rmem = 4096 87380 16777216

# Controls TCP sliding send window buffer (bytes)
net.ipv4.tcp_wmem = 4096 65536 16777216

Ross S. W. Walker
Information Systems Manager
Medallion Financial, Corp.
437 Madison Avenue
38th Floor
New York, NY 10022
Tel: (212) 328-2165
Fax: (212) 328-2125
WWW: http://www.medallion.com <http://www.medallion.com> 

This e-mail, and any attachments thereto, is intended only for use by
the addressee(s) named herein and may contain legally privileged
and/or confidential information. If you are not the intended recipient
of this e-mail, you are hereby notified that any dissemination,
distribution or copying of this e-mail, and any attachments thereto,
is strictly prohibited. If you have received this e-mail in error,
please immediately notify the sender and permanently delete the
original and any copy or printout thereof.