Unable to configure & validate vSAN Stretched Cluster
Article ID: 392876
Updated On:
Products
Issue/Introduction
vSAN Stretch Cluster formation is failing.
Environment
vSAN 7.0x
vSAN 8.0x
Cause
Unable to configure vSAN cluster . This is due to missing networking requirements. In other words, the Underlying Network between the vSAN ESXI data hosts and the witness appliance host do not meet the requirements.
Resolution
Troubleshooting the Underlying Network may require the involvement of the local LAN/WAN administration team.
The Administrator needs to determine why the witness can not communicate with the ESXi data nodes in the vSAN Cluster.
Starting with the 1 witness node and 1 data node, putty/SSH into each to begin the process of testing the network connectivity between the two endpoints.
Use the vCenter UI to gather the VMKernel port (vmk0 for example) of the witness and VMkernel port for vSAN on the Data ESXi host.
Witnesses standardly use the vmkernel for management to move traffic across the WAN.
Data nodes use the VMkernel port IP for vSAN.
The next step once you have gathered the vmkernel port IP address for the witness and the VMkernel port IP for the data node vSAN VMKernel port is to test connectivity.
Use the vmkping command to test the connectivity between witness and data node.
It is required to test in both directions. I.E. from the witness to the data node and from the data node to the witness.
vmkping -I vmk1 -s 1472 <<< IP >>>
vmk1 = the vmkernel you are sending from
-s 1472 = size of MTU
IP = the destination IP
If both directions ping are successful then you have completed the first step of determining if the hosts can communicate simply. You can then move onto testing network ports.
However, if you cannot ping between the two endpoints. Then there is physical networking problem. I.E. there could be firewall rules blocking traffic, or there is could be no route between the IP addresses, or an incorrect VLAN.
Contact you network administrator and share the exact results of the failed vmkping.
Troubleshooting network ports:
If vmkping tests are successful but still the witness is "partitioned". The next step in resolving the issue is to troubleshoot network ports. To test if the required VSAN Ports are open bi-directionally between the data node and the witness use the netcat and TCP dump tool.
Here is a link to the vSAN ports required.
use Netcat and TCP dump:
Netcat -
nc -zv <ip> <port>
TCP dump -
tcpdump-uw -i vmkX | grep <witness IP/FQDN> (vmkX is the vmk for witness traffic and should be VMK0 for management vmkernel)
Ensure you can see communications between the witness IP. Then reverse this on the data node. Communication must be bi-directional.
If you cannot pass traffic on the required ports this is what is preventing the communication between the endpoints.
Please contact your networking administrator and share the output of the Netcat and TCP dump results and explain that the required TCP and UDP ports are required to be open for successful connectivity between the vSAN witness ESXi host and the Data ESXi vSAN Hosts.
Additional Information
Feedback
