Failed to connect to server (code: 1006)
".2020-04-16T08:18:57Z 2020-04-16 08:18:57.986 109 DEBUG nova.objects.console_auth_token [req-########-####-####-####-############ - - - - -] Validated token - console connection is ConsoleAuthToken(access_url_base='https://10.13.0.96:6090/vnc_auto.html',console_type='novnc',created_at=2020-04-16T08:18:57Z,host='ESXi.acme.local',id=147,instance_uuid=########-####-####-####-############,internal_access_path='{"ticket": "########-####-####-####-############", "thumbprint": "####", "cfgFile": "/vmfs/volumes/########-####-####-####.vmx"}',port=902,token='***',updated_at=None) validate /usr/lib/python2.7/site-packages/nova/objects/console_auth_token.py:158
2020-04-16T08:18:58Z 2020-04-16 08:18:58.128 109 INFO nova.compute.rpcapi [req-########-####-####-####-############ - - - - -] Automatically selected compute RPC version 5.1 from minimum service version 37
2020-04-16T08:20:17Z 2020-04-16 08:20:17.767 109 INFO nova.console.websocketproxy [req-########-####-####-####-############ - - - - -] handler exception: [Errno 104] Connection reset by peer
2020-04-16T08:18:58Z vmauthd[4203806]: Connect from remote socket (10.13.0.98:41926).
2020-04-16T08:18:58Z vmauthd[4203806]: Connect from 10.13.0.98
2020-04-16T08:20:17Z vmauthd[4203806]: SSL: syscall error 110: Connection timed out
2020-04-16T08:20:17Z vmauthd[4203806]: recv() FAIL: 110.
2020-04-16T08:20:17Z vmauthd[4203806]: VMAuthdSocketRead: read failed. Closing socket for reading.
2020-04-16T08:20:17Z vmauthd[4203806]: Read failed.
6.x, 7.x
This issue is caused by the communication issue between nova-mks and ESXi vmkernel port because MKS communication is used for displaying VM console.
For example, if MTU of vmkernel port is set to "9,000", nova-mks and vmkernel port tries to use "MSS=8960" for TCP communication. However, MTU configurations of Openstack controller nodes are "1,500" by default. As a result, packet with larger size than 1500 are dropped at vNIC of an Openstack controller node running nova-mks. In the above situation, TCP 3-way handshake can be established, but following server hello packet for SSL handshake is dropped at vNIC.
This issue can be fixed by troubleshooting network issues.
For example, this issue can be fixed by matching MTU among vmkernel port and Openstack controller nodes.