Unable to Create Network Profile in HCX due to, "The proxy server received an invalid response from an upstream server"

Products

• VMware HCX

Issue/Introduction

When attempting to create Network Profiles in VMware HCX (particularly in versions before 4.10), users may encounter a 502 Proxy Error with the message "The proxy server received an invalid response from an upstream server," even in environments without proxy configuration or site pairing. This error prevents the creation of network profiles and blocks the deployment of HCX functionality.

The error message appears as

502 Proxy Error

The proxy server received an invalid response from an upstream server.

The proxy server could not handle the request

Reason: Error reading from remote server

Environment

• HCX Version: Affects versions before 4.10
• vCenter
• NSX-T: Required for overlay network configuration

Cause

This issue occurs due to multiple environmental and configuration factors, particularly in versions before 4.10 where requirements were more strict

1. Missing or incomplete NSX-T overlay network configuration prior to HCX deployment
2. Service initialization timing affecting network profile creation
3. Interaction between HCX services during initial setup and first reboot
4. Environmental dependencies that must be properly configured

The error message suggesting a proxy issue is misleading, as the root cause is related to service initialization and NSX-T configuration requirements rather than actual proxy connectivity problems.

Resolution

Primary Resolution (Recommended)

1. Upgrade to HCX 4.10.3 or later, which handles service initialization more effectively
2. Ensure NSX-T overlay networks are properly configured
3. After verifying all prerequisites below, configure any remaining required components for the environment

Before attempting any resolution, verify these critical environmental prerequisites

1. Time Synchronization
   • Verify NTP configuration on all HCX managers
   • Confirm no TIME_ERROR reports in kernel logs
   • Ensure no time drift between systems
2. DNS Resolution
   • Verify all DNS records are resolving correctly
   • Test forward and reverse DNS lookup functionality
   • Confirm no name resolution issues exist
3. Network Connectivity
   • Test TCP 443 connectivity between components using curl commands
   • Verify no firewall restrictions are in place
   • Confirm routing is properly configured
   • Check for any MTU issues
4. Authentication
   • Verify SSO configuration is working
   • Ensure correct vCenter credentials are being used
   • Confirm authentication services are operational
5. System State
   • Verify all services are running as expected
   • Check for any failed system components
   • Confirm system logs show no critical errors

If upgrade is not immediately possible

1. After verifying all prerequisites above, ensure NSX-T overlay networks are properly configured before HCX deployment
2. Create network profiles immediately after deployment before any service restarts
3. Complete remaining configuration as needed
4. Redeploy with latest version of HCX and follow procedure if all else fails
5. Reach out to Broadcom support if issues continue after verifying each part of your configuration

Additional Information

Versions of HCX before 4.10 had stricter requirements around service initialization and configuration timing, making this issue more likely to occur. The issue has been addressed in HCX 4.10 and later versions, with improved handling of service initialization.

Proper NSX-T configuration remains a prerequisite for HCX network profile creation across all versions.