On Sat, 28 May 2022 15:43:12 +0200 Erik Frangež via CentOS wrote:

we are setting NFS server on CentOS7 system. Everything working OK except speed, speed over NFS very drop... if we run dd command directly on server we are getting speed around 1,4Gbps, if we run from client connected to NFS is 200Mbps.

Do you have maybe some advice what we need to check?

Speed of network card in the server. Speed of network card in the client. Speed of any and all routers and switches between the server and the client.

Am 28.05.22 um 18:40 schrieb Frank Cox:

On Sat, 28 May 2022 15:43:12 +0200 Erik Frangež via CentOS wrote:

...

we are setting NFS server on CentOS7 system. Everything working OK except speed, speed over NFS very drop... if we run dd command directly on server we are getting speed around 1,4Gbps, if we run from client connected to NFS is 200Mbps.

Do you have maybe some advice what we need to check?

Speed of network card in the server. Speed of network card in the client. Speed of any and all routers and switches between the server and the client.

and what config is in place ?

-- Leon

NFS4.x works best with newer kernels. Maybe you could try with an elrepo kernel.

Increase the number of NFS threads on the NFS server (default is 8, which is too little).

|RPCNFSDCOUNT=32|

This is an example of kernel tunning for 10Gb network, adjust to your setup:

vm.min_free_kbytes = 1048576

vm.swappiness = 10

net.core.somaxconn = 4096

# set max buffers to 64MB net.core.rmem_max = 67108864 net.core.wmem_max = 67108864

# increase Linux autotuning TCP buffer size to 32MB net.ipv4.tcp_rmem = 4096 87380 33554432 net.ipv4.tcp_wmem = 4096 65536 33554432

net.core.netdev_max_backlog = 250000

net.ipv4.neigh.default.unres_qlen = 100

net.ipv4.neigh.ens3f0.unres_qlen = 100

net.ipv4.neigh.ens3f1.unres_qlen = 100

fs.file-max = 98584540

You can calculate the buffer size with this tool:

https://www.switch.ch/network/tools/tcp_throughput/?do+new+calculation=do+ne... https://www.switch.ch/network/tools/tcp_throughput/?do+new+calculation=do+new+calculation

Enable readahead for the filesystem for instance:

blockdev --setra 16384 /dev/mapper/mpatha1

for x in sd[a-d]; do blockdev --setra 16384 /dev/$x ; done

It also helps if you disable flowcontrol on the network cards (and switches).

You could also try to reduce the maximum flow rate on the NFS host (set max rate to 8gbit for example)

It would be best to test the actual speed by iperf3 test as someone else already suggested. But since the performance it so bad, I would suspect that the source of the problem is in the the network, not the server.

Cheers,

Barbara

On 5/29/22 01:52, Valeri Galtsev wrote:

On 5/28/22 9:43 AM, Erik Frangež via CentOS wrote:

...

Hi guys,

we are setting NFS server on CentOS7 system. Everything working OK except speed, speed over NFS very drop... if we run dd command directly on server we are getting speed around 1,4Gbps, if we run from client connected to NFS is 200Mbps.

Do you have maybe some advice what we need to check?

Make sure you have good speed of physical connection along the whole path from server to client(s), enable Jumbo packets on all switches along the path. NFS experts will add NFS specific tuning.

Valeri

...

Thank you!

Best, Erik

---

CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos

---

CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos