Creating vSAN clusters up to 64 nodes

Products

• VMware vSAN

Issue/Introduction

With vSAN 6.0 and later releases, you can create a cluster of more than 32 hosts, up to a maximum of 64 hosts in a vSAN cluster.

Environment

VMware vSAN 6.1.x
VMware vSAN 7.0.x
VMware vSAN 6.2.x
VMware vSAN 6.7.x
VMware vSAN 6.5.x
VMware vSAN 6.6.x
VMware vSAN 6.0.x

Resolution

Steps for vSAN 7.0U3d and later:
The large cluster support is enabled by default. The user does not need to perform any additional configuration steps to configure vSAN clusters with more than 32 nodes or extend existing clusters beyond 32 nodes.

Steps for vSAN 6.x clusters:

Note: These steps are needed for vSphere versions older than 6.7U3. If a vSphere cluster is upgraded to 6.7U3 or higher, the user should use UI/API to configure large-scale clusters.

 

For vSAN version below 6.7U3
To set the advanced configuration options:

1. Open an SSH session to the first ESXi host.
2. Configure the advanced settings for increased node support on each host in the cluster:
   
   esxcli system settings advanced set -o /VSAN/goto11 -i 1
    
3. To confirm the change took affect run: esxcfg-advcfg -g /VSAN/goto11
4. Increase the TCP/IP heap size:
   
   esxcli system settings advanced set –o /Net/TcpipHeapMax –i 1536
    
5. For early build vSphere 6.0 hosts (hosts running ESXi builds prior to ESXi 6.0p01 (build 2809209), you must also set the client limit to 65 to allow up to 64 hosts:
   
   esxcli system settings advanced set -o /CMMDS/clientLimit 65
    
6. Put the ESXi host in maintenance mode and restart.
   
   Note: Use the Ensure Accessibility option.
    
7. Exit maintenance mode.
8. Repeat steps #1 to #6 for each host in the cluster.

Steps for 6.7U3 and higher but below version 7.0U3d: