Hi Rudi,

Does anyone have, or know of a comparison chart of the different network adapters, i.e. 1GB / 10GB, Infiniband, etc. And if possible with a few top brand NIC's and switches listed as well.

I would like to see, for example, what the max throughput is of a 1GB NIC (and this could probably differ from PCI to PCIE-x1 to PCIE-x4), and 10GBE. Different switches would probably also have different ratings, but could a layer 2 switch & layer3 switch deliver the same performance for example?

Basically I need to know what upload / download speeds I should be getting from the different networks, set aside other options like CPU / RAM / disc IO / etc.

If you exclude the host capability to deliver data to the interface and also the ability of the host to assemble and disassemble packets, then the speeds will be dependent on the switches capability to handle the packet size and numbers of packets arriving at an interface in a specific time period. You should expect wirespeeds (minus the overhead of the Ethernet frame and the IP frame) for most switches with usual packet sizes (below 1500 bytes) i.e. the usual (if there is such a thing) packet size, provided that you're not flooding the interfaces with very small packets and your switch is set to store and forward. If your switch is able to cut and forward which is a must for jumbo frames to be handled quickly, then you can also expect close to wirespeeds for any frame size. Cut and forward switches are expensive but are a must for storage networks if you're interested in low latency switching.

Layer 3 switching is basically routing done on a switch and therefore increases the latency slightly to a lot as the switch has to decode the layer 3 information and make a decision based on that as opposed to the outer layer 2 information.

Due to the many factors affecting network speeds, most switch manufacturers specify the switch capability/capacity in backplane bandwidth, maximum packet numbers switched per second and memory available for store and forward. As soon as one of these limits is breached then the performance will take a hit and this hit can be a big one.

I'm sorry that I can't be more helpful and provide you with what you're after, but I hope that this has answered some questions for you.

Rgds

Simon.

On Friday 22 October 2010, Rudi Ahlers wrote:

Hi all,

Does anyone have, or know of a comparison chart of the different network adapters, i.e. 1GB / 10GB, Infiniband, etc. And if possible with a few top brand NIC's and switches listed as well.

Iperf(tcp) Good 1G eth: 945 Mbps Good 10G eth(w/o tcp tuning): 6-7 Gbps Same 10G with some tcp tuning): 9.4 Gbps

Infiniband(MPI or raw verbs): SDR: 980 MB/s DDR: 1900 MB/s QDR: 3700 MB/s

I don't have numbers for IPoIB (tcp on Infiniband) since we dont use that much.

I would like to see, for example, what the max throughput is of a 1GB NIC (and this could probably differ from PCI to PCIE-x1 to PCIE-x4), and 10GBE. Different switches would probably also have different ratings,

Most or all switches I've used the last few year can take wirespeed 1G eth on atleast a few ports concurrently. For 10G eth I don't know much more than that our procurve, cisco and bladenetworks equipment can do wire speed (atleast on a few ports concurrently).

Infiniband swtiches are typically very good at delivering bandwidth (as long as you avoid congestion).

but could a layer 2 switch & layer3 switch deliver the same performance for example?

As previously stated by another poster, layer3 is only features, says nothing about performance (although features tend to cost you..).

Basically I need to know what upload / download speeds I should be getting from the different networks, set aside other options like CPU / RAM / disc IO / etc.

All of my figures above assumes "good hardware". If you want to push 10G or more you'll need PCI-express gen2 and a modern CPU/memory.

/Peter