lhecking at users.sourceforge.net wrote: > I have a Dell server with four bonded, gigabit interfaces. Bonding mode is > 802.3ad, xmit_hash_policy=layer3+4. When testing this setup with iperf, > I never get more than a total of about 3Gbps throughput. Is there anything > to tweak to get better throughput? Or am I running into other limits (e.g. > was reading about tcp retransmit limits for mode 0). > > The iperf test was run with iperf -s on the server, and iperf -c server > on four clients connected to the same switch. > > ------------------------------------------------------------ > Server listening on TCP port 5001 > TCP window size: 85.3 KByte (default) > ------------------------------------------------------------ > [ 4] local 10.10.15.193 port 5001 connected with 10.10.15.184 port 48588 > [ 5] local 10.10.15.193 port 5001 connected with 10.10.15.187 port 49231 > [ 6] local 10.10.15.193 port 5001 connected with 10.10.15.188 port 53197 > [ 7] local 10.10.15.193 port 5001 connected with 10.10.15.189 port 55309 > [ ID] Interval Transfer Bandwidth > [ 4] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec > [ 5] 0.0-10.0 sec 872 MBytes 728 Mbits/sec > [ 6] 0.0-10.0 sec 318 MBytes 267 Mbits/sec > [ 7] 0.0-10.0 sec 1.10 GBytes 939 Mbits/sec > [ 8] local 10.10.15.193 port 5001 connected with 10.10.15.184 port 48589 > [ 4] local 10.10.15.193 port 5001 connected with 10.10.15.187 port 49234 > [ 5] local 10.10.15.193 port 5001 connected with 10.10.15.188 port 38864 > [ 6] local 10.10.15.193 port 5001 connected with 10.10.15.189 port 55311 > [ 8] 0.0-10.0 sec 1.10 GBytes 941 Mbits/sec > [ 4] 0.0-10.0 sec 862 MBytes 721 Mbits/sec > [ 5] 0.0-10.0 sec 322 MBytes 270 Mbits/sec > [ 6] 0.0-10.0 sec 1.10 GBytes 939 Mbits/sec Could it be that with the combinations of IP addresses and port numbers used by iperf above, it ends up with only 3 of the links being used? According to the Linux bonding docs, xmit_hash_policy=layer3+4 uses: ((source port XOR dest port) XOR ((source IP XOR dest IP) AND 0xffff) modulo slave count So I guess you could plug in in the above IP addresses and port numbers and see if you get 3 unique results? James Pearson