Launching VM Web Console fails with "Couldn't establish a connection to the VM web console" message after upgrading to vSphere 9.0

Products

• VMware vCenter Server

Issue/Introduction

• After upgrading vCenter Server and ESXi hosts to version 9.0, launching web console of any VM from vCenter Server fails with the following error : "Couldn't establish a connection to the VM web console." 
  
• vCenter Server and ESXi hosts are assigned with Custom CA certificates. 
• ESXi certificate is validated to be correctly formatted as per KB : Couldn't establish a connection to the VM web console.
• The envoy logs on the vCenter Server reports the following error indicating a failure to trust the ESXi host's certificate. 
  Log location: /var/log/vmware/envoy/envoy.log
  
  YYYY-MM-DDTHH:MM:SS error envoy[21977] [Originator@6876 sub=connection] Failed to load trusted CA certificates from <inline>
YYYY-MM-DDTHH:MM:SS info envoy[22005] [Originator@6876 sub=connection] [Tags: "ConnectionId":"415342"] remote address:<ESXi Host IP Address where the VM is running>:443,TLS_error:|268435581:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED:TLS_error_end-----BEGIN CERTIFICATE-----
<ESXi Host Machine SSL certificate>
-----END CERTIFICATE-----

Environment

• VMware vCenter Server 9.0
• VMware vSphere ESX 9.0

Cause

When Custom CA certificates are in use, the Envoy service may fail to correctly load or validate the inline trusted CA chain required to proxy the secure WebSocket connection (WSS) from the browser to the ESXi host. Consequently, the connection is terminated by the proxy before it reaches the VM.

Resolution

This is a known issue in VCF 9.x release and is expected to get fixed in the upcoming releases. 

As a workaround, enable the MKS Dev Proxy configuration within vCenter Server Advanced Settings : 

1. Login to vCenter Server > Click on vCenter Server instance. 
2. Select the Configure tab > Advanced Settings > Edit Settings. 
3. In the Name field, enter the following configuration "config.mksdevproxy.enable" and set the value to "true" and click on ADD and then Save like below: 
   
4. Try opening the VM Web console from vCenter Server now. 

Additional Information

Couldn't establish a connection to the VM web console