VMware Cloud Director UI and API access fails with ERR_CONNECTION_RESET after configuring Public Address

Products

• VMware Cloud Director

Issue/Introduction

• Users are experiencing ERR_CONNECTION_RESET and "loading chunk [number] failed" errors when attempting to access the VMware Cloud Director 10.6 HTML5 UI or Cloud API. This typically occurs after a Public Address has been configured for the environment.
• When observing the browser developer tools (F12) Network tab, you will see network requests for .js chunks failing with ERR_CONNECTION_RESET or a ChunkLoadError. Additionally, load balancer or firewall logs may show HTTP requests being dropped or TCP connections being terminated due to header limits.

Environment

• Product: VMware Cloud Director 10.6

• Network Appliance: External Load Balancers or Firewalls (e.g., F5 LTM)

Cause

The issue is caused by HTTP response headers generated by the Cloud Director OpenAPI exceeding the default maximum header size and count limits on the network appliance. Configuring a Public Address in Cloud Director converts relative API links to absolute URLs, which substantially increases the HTTP header payload. When these headers exceed the appliance's profile limits, the load balancer or firewall terminates the connection. 

Resolution

To resolve this issue, you must increase the HTTP header limits on your external load balancer or firewall to accommodate the larger OpenAPI payload.

For F5 LTM appliances: Modify the HTTP profile associated with the Cloud Director virtual server using the following configuration:

ltm profile http <profile_name> {
    enforcement {
        max-header-count 256
        max-header-size 65535
    }
}

For other network appliances: Ensure the maximum header count is set to at least 256 and the maximum header size is increased to 65535 bytes (64 KB).

The settings are also in the configuration guide for Load BalancingCloud Director:

VMware Cloud Director Cell Load Balancing Configuration

Additional Information

• This behavior is also seen when performing the following tasks:

  • Global Roles Cannot be Created or Edited in the Cloud Director HTML5 UI

  • "ERROR: Unexpected token "<", "<!DOCTYPE" .. is not valid JSON" when Viewing Edge Gateway NAT Rules

• It is recommended to Subscribe to this knowledge article to get updates on this issue.

  • Subscribe to a Broadcom Knowledge Article.