vSAN cluster network partitioned after updating certificates

Products

• VMware vSAN

Issue/Introduction

• vSAN cluster partitioned
• Objects/VMs inaccessible
• Datastore capacity not showing expected size
• Expired certificates in cluster for prolonged period of time

Environment

VMware vSAN (All Versions)

Cause

Due to failed certificate updates from self signed to custom certificates on all hosts in the cluster, this results in vCenter not trusting/verifying the host certificates for the hosts, which resulted in 2 of the hosts getting removed from the vSAN unicast agent list. The hosts were removed due to vCenter being authoritative for pushing vSAN updates to ESXi.

Resolution

Confirm vCenter is still authoritative from all hosts in the vSAN cluster by running the following:

    esxcfg-advcfg -g /VSAN/IgnoreClusterMemberListUpdates

Value of 0 means vCenter is authoritative.

Follow KB Configuring vSAN Unicast networking from the command line, to manually add the missing host's unicast agent list entries to each of the affected hosts.