On Thu, 21 Jan 2021 23:33:56 +0000 lejeczek via CentOS centos@centos.org wrote:
Hi guys.
Hoping some net experts my stumble upon this message, I have an IPoIB direct host to host connection and:
...
$ iperf3 -c 10.5.5.97
[ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 16.2 GBytes 13.9 Gbits/sec 0 sender [ 5] 0.00-10.00 sec 16.2 GBytes 13.9 Gbits/sec receiver
It's rather an oldish platform which hosts the link, PCIe is only 2.0 but with link of x8 that should be able to carry more than ~13Gbits/sec. Infiniband is Mellanox's ConnectX-3.
If you want to test the infiniband performance you ib_write_bw for example not iperf.
IPoIB will always be quite a bit slower than native IB.
That said, if you want to optimize IPoIB for performance make sure you're running connected mode not datagram mode.
/Peter