CNS/FCD Deployment fail due to vSAN cluster health discrepancy between UI Health and CLI Partition

Products

• VMware vSAN

Issue/Introduction

• vSAN Stretched or 2-node cluster reports a partition error: A general system error occurred: VSAN Cluster is in partition.
• Attempts to create VMware Cloud Native Storage (CNS) or CSI volumes fail with errors: Failed to init fcd-catalog for datastore.
• vSAN Health UI may report the cluster as 100% healthy, but CLI command esxcli vsan cluster get shows a Sub-Cluster Member Count of less than 3.
• The esxcli vsan cluster unicastagent list or cluster membership shows "orphaned hosts" or unknown Node IDs associated with the Witness.

Environment

• VMware vSAN 7.x / 8.x
• Stretched Clusters or 2-node Clusters

Cause

This issue occurs when the vSAN Witness appliance metadata becomes stale or corrupted, often listing entries for unrelated clusters or orphaned nodes. This prevents the formation of a healthy quorum even if network connectivity is valid.

Resolution

To resolve the partition and restore FCD catalog initialization, replace the Witness appliance with a fresh deployment:

1. Verify Networking: Ensure UDP ports 12321, 12345, and 2233 are open between data nodes and the witness.
2. Deploy New Witness: Download and deploy a new Witness appliance OVA that matches the ESXi build version of the data nodes.
3. Replace Witness Host:
   • Log into vCenter.
   • Select the vSAN Cluster > Configure > vSAN > Fault Domains.
   • Click Change Witness Host.
   • Follow the wizard to select the new Witness appliance.
4. Validate: Run esxcli vsan cluster get on a data node. The Sub-Cluster Member Count should return to 3 and the fcd-catalog should initialize automatically.

If assistance is required, see Contact Support.