Direct connect vSAN 2-node stretch Cluster showing basic unicast connectivity error in skyline health and is network partitioned

Products

• VMware vSAN

Issue/Introduction

Direct connect 2-node stretch cluster is showing as partitioned, despite no networking changes after a host reboot or a cluster shutdown and restart.

Environment

VMware vSAN (All Versions)

2-node stretch cluster

Cause

With direct connect 2-node stretch clusters, if NIC teaming and failover settings have multiple uplinks set to active, it's possible for each host to power back up using different vmnics. This will result in the directly connected hosts to not be able to communicate and partition, since vSAN only uses one NIC

Resolution

There are 2 options to correct this issue a resolution and a workaround. 

Resolution

Change NIC teaming and failover settings to Active/Standby so both hosts use the same vmnic as Active, and a same alternate vmnic as standby. This will ensure that the hosts are always utilizing the same vmnic and will avoid this issue.  Please see: NIC teaming in ESXi and ESX and process to configure

Workaround. 

This is a temp solution and it can reoccur on any host reboot. 

If you are not able to convert the networking from Active/Active to Active/Standby. You can validate which vmnics are being utilized via "esxtop" option N on both hosts.  Once you have identified the current vmnic in use on both hosts.  Down the active vmnic on one host  to force the traffic across the same connection. 

esxcli network nic down -n <vmnicx>

Once you have validated that the vSAN cluster has formed please up the vmnic you downed. 

 esxcli network nic up -n <vmnicx>