VM web console fails with Error "Couldn't establish a connection to the VM web console"

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

Error "Couldn't establish a connection to the VM web console" when accessing VM web console for VMs on Enhanced linked mode vCenter server
After upgrading esxi to 8.0 VM web console stops working

Vpxd Logs may show errors as below:

YYYY-MM-DDTHH:MM:SS error vpxd[06972] [Originator@6876 sub=Default opID=1c0c88df] [VpxLRO] -- ERROR lro-16328927 -- 52ed0607-0731-8931-4fea-03adcdff21ce(52bba5f6-c1ed-e399-d077-385b00678c12) -- vm-XXXXX -- vim.VirtualMachine.acquireTicket: :vmodl.fault.SystemError
--> Result:
--> (vmodl.fault.SystemError) {
-->    faultCause = (vim.fault.GenericVmConfigFault) {
-->       faultCause = (vmodl.MethodFault) null,
-->       faultMessage = (vmodl.LocalizableMessage) [
-->          (vmodl.LocalizableMessage) {
-->             key = "msg.vigor.transport.connection.error",
-->             arg = <unset>,
-->             message = "Disconnected from virtual machine."
-->          },
-->          (vmodl.LocalizableMessage) {
-->             key = "msg.vigor.transport.connection.fail2",
-->             arg = <unset>,
-->             message = "Failed to establish transport connection."
-->          },
-->          (vmodl.LocalizableMessage) {
-->             key = "msg.asyncsocket.remotedisconnect",
-->             arg = <unset>,
-->             message = "Remote disconnected"
-->          }
-->       ],
-->       reason = "Disconnected from virtual machine."
-->       msg = "Disconnected from virtual machine."
-->    },
-->    faultMessage = <unset>,
-->    reason = "Undeclared fault"
-->    msg = "Received SOAP response fault from [<<io_obj p:0x00007f04f4b929e8, h:132, <UNIX ''>, <UNIX '/var/run/envoy-hgw/hgw-pipe'>>, /hgw/host-XX/vpxa>]: acquireTicket
--> Received SOAP response fault from [<<io_obj p:0x000000f32c4ecbe0, h:21, <TCP '127.0.0.1 : 59862'>, <TCP '127.0.0.1 : 8307'>>, /sdk>]: acquireTicket
--> Method vim.VirtualMachine.acquireTicket threw undeclared fault of type vim.fault.GenericVmConfigFault"
--> }
--> Args:
-->
--> Arg ticketType:
--> "webmks"

/var/log/vmware/envoy/envoy-access.log

[timestamp] info envoy[2538] [Originator@6876 sub=Default] [timestamp] GET /ui/webconsole.html?vmId=vm-#######&vmName=VMNAME&numMksConnections=0&serverGuid=83######-####-####-####-######df0a22&locale=##-## 200 via_upstream - 0 4279 zstd 1 1 0 ##.##.##.##:49287 HTTP/2 TLSv1.2 Source vCenter IP:443 127.0.0.1:55988 HTTP/1.1 - 127.0.0.1:5090 - - [timestamp] info envoy[2538] [Originator@6876 sub=Default] [timestamp] GET /ui/##########-###### 200 via_upstream - 0 144 zstd 62 62 0 ##.##.##.##:49287 HTTP/2 TLSv1.2 Source vCenter IP:443 127.0.0.1:55908 HTTP/1.1 - 127.0.0.1:5090 - - [timestamp] info envoy[2538] [Originator@6876 sub=Default] [timestamp] GET /##########-######/###############/7af47c4ab6b720de 503 no_healthy_upstream UH 0 19 - 120009 - - ##.##.##.##:49287 HTTP/2 TLSv1.2 Source vCenter IP:443 - - - ESXi Host IP:443 - -

Environment

vCenter - 8.x
vCenter - 9.x
ESXi - 8.x
ESX - 9.x

Cause

This issue occurs if Port 443 access blocked in esxi firewall for linked mode vCenter.

After vSphere 8.0, port 443 is used to acquire tickets for web console.

Resolution

Connecting to a Browser-Based Virtual Machine Console Through the vSphere Client

Ensure to allow vSphere Client to access vCenter Server on port 443.

Below are the steps to allow linked mode vCenter access in esxi firewall:

Select ESXi host in the inventory
Navigate to Host > Configure > Firewall
Click Edit and search for firewall rule "vSphere Web client"
if the rule is restricted for Ip addresses, switch to ANY
Or add Ip address of Linked mode vCenter server.

Retry opening web console of VMs.