Hello,
does it in any respect (throughput/performance, cpu load, I/O load, resilience, ...) matter, if one mounts subdirectories of an NFS (v3) export into separate directories or if one just mounts the parent directory?
I.e. like this:
server:/export/base/a -> /mnt/a server:/export/base/b -> /mnt/b server:/export/base/c -> /mnt/c server:/export/base/d -> /mnt/d server:/export/base/e -> /mnt/e
or simply like this:
server:/export/base -> /mnt
I would guess, that it doesn matter at all, but at $work the standard setup on all cluster nodes is, to mount >20 subdirectories from the NFS server individually and the justification is, that is more performant and reliable.
Can this be confirmed (or denied)?
I couldn't find this method of "NFS performance tuning" mentioned anywhere and from a management perspective the sheer amount of mounted filesystems makes the list confusing and harder to troubleshoot.
Any thoughts?
Regards frank
On Wed, 27 Jul 2016, Frank Thommen wrote:
Hello,
does it in any respect (throughput/performance, cpu load, I/O load, resilience, ...) matter, if one mounts subdirectories of an NFS (v3) export into separate directories or if one just mounts the parent directory?
I.e. like this:
server: /export/base/a -> /mnt/a server: /export/base/b -> /mnt/b server: /export/base/c -> /mnt/c server: /export/base/d -> /mnt/d server: /export/base/e -> /mnt/e
or simply like this:
server:/export/base -> /mnt
Performance wise, any bottleneck will almost certainly be tied to the disks on the back end, not the nfs process itself.
There are a couple good reasons for splitting up the mounts:
1. They can have different export restrictions (e.g., for different client hosts, ro vs. rw permissions, user squashing).
2. /base/[a-e] live on different RAID arrays and might benefit from different management cycles; that'd also be a case where multiple exports might be a good idea. That said, I've never managed an exported filesystem consisting of different arrays; we've always exported at the RAID level or below.
There is a slight performance related reason for exporting disk partitions individually, the performance boost is server-side as Paul says. The advantage is that the no_subtree_check can be used without any additional security risk.
It is probably the case that the /export/base/a is a partition, is exported with no_subtree_check, and therefore there is a small performance boost.
Preventing server side mount point traversal can also form part of a security mechanism if servers have different security options for different mount points, but in this case mounting server:/export/base wouldn't give you the same client view of the filesystem tree as mounting each individually if it worked at all.
Cheers, Sean
On 27 July 2016 at 23:21, Paul Heinlein heinlein@madboa.com wrote:
On Wed, 27 Jul 2016, Frank Thommen wrote:
Hello,
does it in any respect (throughput/performance, cpu load, I/O load, resilience, ...) matter, if one mounts subdirectories of an NFS (v3) export into separate directories or if one just mounts the parent directory?
I.e. like this:
server: /export/base/a -> /mnt/a server: /export/base/b -> /mnt/b server: /export/base/c -> /mnt/c server: /export/base/d -> /mnt/d server: /export/base/e -> /mnt/e
or simply like this:
server:/export/base -> /mnt
Performance wise, any bottleneck will almost certainly be tied to the disks on the back end, not the nfs process itself.
There are a couple good reasons for splitting up the mounts:
They can have different export restrictions (e.g., for different client hosts, ro vs. rw permissions, user squashing).
/base/[a-e] live on different RAID arrays and might benefit from different management cycles; that'd also be a case where multiple exports might be a good idea. That said, I've never managed an exported filesystem consisting of different arrays; we've always exported at the RAID level or below.
-- Paul Heinlein <> heinlein@madboa.com <> http://www.madboa.com/ _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On Wed, Jul 27, 2016 at 11:40:38PM +0100, Sean Brisbane wrote:
There is a slight performance related reason for exporting disk partitions individually, the performance boost is server-side as Paul says. The advantage is that the no_subtree_check can be used without any additional security risk.
It is probably the case that the /export/base/a is a partition, is exported with no_subtree_check, and therefore there is a small performance boost.
Preventing server side mount point traversal can also form part of a security mechanism if servers have different security options for different mount points, but in this case mounting server:/export/base wouldn't give you the same client view of the filesystem tree as mounting each individually if it worked at all.
Cheers, Sean
On 27 July 2016 at 23:21, Paul Heinlein heinlein@madboa.com wrote:
On Wed, 27 Jul 2016, Frank Thommen wrote:
Hello,
does it in any respect (throughput/performance, cpu load, I/O load, resilience, ...) matter, if one mounts subdirectories of an NFS (v3) export into separate directories or if one just mounts the parent directory?
I.e. like this:
server: /export/base/a -> /mnt/a server: /export/base/b -> /mnt/b server: /export/base/c -> /mnt/c server: /export/base/d -> /mnt/d server: /export/base/e -> /mnt/e
or simply like this:
server:/export/base -> /mnt
Performance wise, any bottleneck will almost certainly be tied to the disks on the back end, not the nfs process itself.
There are a couple good reasons for splitting up the mounts:
They can have different export restrictions (e.g., for different client hosts, ro vs. rw permissions, user squashing).
/base/[a-e] live on different RAID arrays and might benefit from different management cycles; that'd also be a case where multiple exports might be a good idea. That said, I've never managed an exported filesystem consisting of different arrays; we've always exported at the RAID level or below.
Exporting them individually also prevents the remote system from accessing /mnt/[!abcde] that you did not intend to make available.
Jon
On the server side, the export is defined for /export/base, not for /export/base/x. But I see the points. It seems, that we should probably revisit our export/mount setup :-)
frank
On 07/28/2016 12:40 AM, Sean Brisbane wrote:
There is a slight performance related reason for exporting disk partitions individually, the performance boost is server-side as Paul says. The advantage is that the no_subtree_check can be used without any additional security risk.
It is probably the case that the /export/base/a is a partition, is exported with no_subtree_check, and therefore there is a small performance boost.
Preventing server side mount point traversal can also form part of a security mechanism if servers have different security options for different mount points, but in this case mounting server:/export/base wouldn't give you the same client view of the filesystem tree as mounting each individually if it worked at all.
Cheers, Sean
On 27 July 2016 at 23:21, Paul Heinlein heinlein@madboa.com wrote:
On Wed, 27 Jul 2016, Frank Thommen wrote:
Hello,
does it in any respect (throughput/performance, cpu load, I/O load, resilience, ...) matter, if one mounts subdirectories of an NFS (v3) export into separate directories or if one just mounts the parent directory?
I.e. like this:
server: /export/base/a -> /mnt/a server: /export/base/b -> /mnt/b server: /export/base/c -> /mnt/c server: /export/base/d -> /mnt/d server: /export/base/e -> /mnt/e
or simply like this:
server:/export/base -> /mnt
Performance wise, any bottleneck will almost certainly be tied to the disks on the back end, not the nfs process itself.
There are a couple good reasons for splitting up the mounts:
They can have different export restrictions (e.g., for different client hosts, ro vs. rw permissions, user squashing).
/base/[a-e] live on different RAID arrays and might benefit from different management cycles; that'd also be a case where multiple exports might be a good idea. That said, I've never managed an exported filesystem consisting of different arrays; we've always exported at the RAID level or below.
-- Paul Heinlein <> heinlein@madboa.com <> http://www.madboa.com/ _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On 07/28/2016 12:21 AM, Paul Heinlein wrote:
On Wed, 27 Jul 2016, Frank Thommen wrote:
Hello,
does it in any respect (throughput/performance, cpu load, I/O load, resilience, ...) matter, if one mounts subdirectories of an NFS (v3) export into separate directories or if one just mounts the parent directory?
I.e. like this:
server: /export/base/a -> /mnt/a server: /export/base/b -> /mnt/b server: /export/base/c -> /mnt/c server: /export/base/d -> /mnt/d server: /export/base/e -> /mnt/e
or simply like this:
server:/export/base -> /mnt
Performance wise, any bottleneck will almost certainly be tied to the disks on the back end, not the nfs process itself.
There are a couple good reasons for splitting up the mounts:
- They can have different export restrictions (e.g., for different client hosts, ro vs. rw permissions, user squashing).
I'm not sure you can define individual restrictions for subdirectories of exported filesystems? In our case export permissions are set for "server:/export/base".
- /base/[a-e] live on different RAID arrays and might benefit from different management cycles; that'd also be a case where multiple exports might be a good idea. That said, I've never managed an exported filesystem consisting of different arrays; we've always exported at the RAID level or below.
Agreed, but this is not the case in our situation.
frank
-
Frank Thommen -
Jon LaBadie -
Paul Heinlein -
Sean Brisbane