VCD Portal Inaccessible via Load Balancer after Single Cell Failure

Products

• VMware NSX
• VMware vCloud Director 5.x

Issue/Introduction

• Users report they are unable to access the VMware Cloud Director (VCD) Public Address (Web Console or API).
• The NSX Native Load Balancer is used to distribute traffic to the VCD cells acting as backend servers.
• The Health Monitor is not configured for the Server Pool

 

Environment

VMware NSX

VMware vCloud Director

Cause

The failure is caused by a missing Health Monitor on the NSX Server Pool.

The Failure Logic

1. Algorithmic Bias: The Load Balancer tracks active sessions to distribute load. When a VCD cell fails, its active connection count drops to zero.

2. The "Black-Hole" Effect: The "Least Connection" algorithm identifies the failed cell as the "least busy" member. Consequently, it directs all new incoming traffic to the dead node.

3. Lack of Awareness: Without an active HTTPS probe, the Load Balancer has no mechanism to "prune" the dead cell from the pool. The Virtual Server remains in an UP state, but 100% of the traffic is routed to a non-responsive backend.

Resolution

Configure and attach an active HTTPS Monitor to the Server Pool to ensure the Load Balancer can prune dead nodes from the rotation.

Implementation Steps:

1. Navigate to Networking > Load Balancer > Monitors in the NSX UI.

2. Create a new monitor using the parameters from the link in the "Additional Inforamation".

3. Go to the Server Pool associated with the VCD Cells.

4. Edit the Pool and select the new HTTPS Monitor.

Additional Information

https://techdocs.broadcom.com/us/en/vmware-cis/cloud-director/vmware-cloud-director/10-6/vcd-cell-load-balancing-configuration.html