Increasing the default value that defines the maximum number of NFS mounts on an ESXi/ESX host

Products

• VMware vSphere ESXi

Issue/Introduction

This article describes how to change the default value for the configuration parameter that defines the maximum number of NFS mounts on an ESXi/ESX host.

This article provides information on:

• Configuring NFS TCP and NFS HEAP settings
• Identifying appropriate settings for TcpipHeapSize and TcpipHeapMax.

Note: Older "6.0"updated hosts can have earlier smaller Net.TcpipHeapSize setting in place after update.
          vSphere 6.0 has increased the maximum heap size and added support for the NFS 4.1 protocol. There are some differences between NFS3 and NFS4.1 support in vSphere 6.0. 

Environment

VMware vSphere ESXi 6.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

 

Resolution

The vSphere ESXi/ESX default configuration allows for only eight NFS mounts per ESXi/ESX host. There are three advanced configuration options which control the maximum number of NFS mounts:

To edit advanced configuration options, select the ESXi/ESX host in the Inventory Panel, then navigate to Configuration > Software > Advanced Settings to launch the Settings window. 
Select and highlight any of the advance options listed below the User Interface page will display, the range from Default to the Maximum value that can be set.

Under NFS, Select "NFS.MaxVolumes -  Controls the total number of concurrently mounted NFS datastores:

ESXi/ESX Version	Default	Maximum Supported
3.x	8	32
4.x	8	64
5.0/5.1/5.5	8	256
6.0/6.5/6.7/7.0/8.0	32	256

From vSphere 7.0, the default value for NFS.MaxVolumes is 32, and can be increased up to 256.

Under Net, Select Net.TcpipHeapSize: VMkernel heap memory (in MB) allocated for TCP/IP network connectivity.

ESXi/ESX Version	Recommended Value
3.x	30
4.x	32
5.0/5.1/5.5	32
6.0 and later	Not required to change

 

Under Net, Select Net.TcpipHeapMax:  Maximum allowed VMkernel heap (in MB) for TCP/IP operations.

 

ESXi/ESX Version	Recommended Value
3.x	120
4.x	128
5.0/5.1	128
5.5	512
6.0	1536
6.5/6.7	512 (default)
7.0	1024 (default)
8.0	1024 (default)

 

From ESXi 6.0 onward, the default Net.TcpipHeapMax is typically sufficient for up to 256 mounts using default socket sizes and does not require manual adjustment unless custom socket sizes are configured.   
Changing Net.TcpipHeapSize and/or Net.TcpipHeapMax requires a host reboot for the changes to take effect.

Additional Information

For more information on advanced NFS settings, see Definition of the advanced NFS options and the Best Practices For Running NFS with VMware vSphere whitepaper.
Modify the three advanced configuration options using the vSphere Client or other method. For a full list of methods, see Configuring advanced options for ESXi/ESX.

ESXi 6.5U3, 6.7U3, 7.0 and 8.0:

In these releases, NFSv3 socket sizes are increased. Default socket size is 1MB(earlier 256K) for both send and receive sockets. Max that can be configured is 5MB(earlier 512K).
Default Net.TcpipHeapMax is increased to 1024MB. Max Net.TcpipHeapMax that can be configured is 3200MB.
Following equation gives the TCPIP heap required for number of shares.
TCPIPHeapMax required = shares * (send socket size + receive socket size)

Example 1: 
Shares = 256. Socket size = 1MB (default)
TCPIPHeapMax required = 256 * (1M + 1M) = 512MB
So, for the default socket sizes of 1MB, existing default TCPIPHeapMax of 1024MB is sufficient.

Example 2:
Shares = 256. Configured Socket size = 5MB (NFS.SendBufferSize, NFS.ReceiveBufferSize)
TCPIPHeapMax = 256 * (5M + 5M) = 2560MB
In this case, one needs to increase the TCPIPHeapMax to atleast 2560MB.
These settings enable the maximum number of NFS mounts for vSphere ESXi/ESX.
Changing Net.TcpipHeapSize and/or Net.TcpipHeapMax requires a host reboot for the changes to take effect.

Possible Log Errors:   2020-01-24T23:34:06.268Z cpu0:2099888 opID=4c099e1c)NFS: 205: NFS mount 1x.xxx.xxx.xx:/nfs_datastore failed: NFS has reached the maximum number of supported volumes