Inter-node communication failure and Fleet Manager import failure in VCF Operations following the addition of a Witness node

Products

• VCF Operations

Issue/Introduction

The issue occurred when attempting to deploy a Continuous Availability (CA) architecture. CA stretches the VCF Operations cluster across two fault domains, using a third independent Witness node as a tiebreaker.

Environment

VCF Operations 9.0x

Cause

Deploying a Witness node in the cluster consumed the remaining available vSphere host license capacity.

This left the production ESXi host hosting the replica VM unlicensed, causing vCenter to force the host into a disconnected state.

With the replica VM isolated, VCF Operations entered split-brain defense mode (Degraded status), which caused Fleet Manager's active health checks to fail the environment import.

Resolution

Additional vSphere host licenses were procured. However, to ensure cluster integrity and avoid lingering database sync issues, the customer elected to perform a clean teardown and greenfield rebuild of the VCF Operations environment.

Additional Information

Prior to re-running the VCF Operations deployment wizard, ensure that the newly procured vSphere licenses are actively assigned to all target ESXi hosts (including the Witness host). If any host lacks an active entitlement during deployment, the Witness node will fail power-on validation and trigger cluster isolation.