Unable to deploy Service Mesh with Error: Operation timedout in state POST_CONFIG_VIX

Products

• VMware HCX
• VMware Cloud on AWS

Issue/Introduction

• Unable to deploy Service Mesh with Error: Operation timedout in state POST_CONFIG_VIX.
• ESXi has 2 management vmk with different network segments.
• The IP address that is resolved by the vCenter DNS record is different than the IP address that is resolved by the HCX DNS record.

Cause

When creating a Service Mesh, the HCX manager tries to connect to ESXi's Management vmk where the Fleet appliances will be deployed on.
Before that, HCX manager retrieves the destination information from vCenter.
 
The destination information propagated from vCenter includes the IP address of ESXi's Management vmk that is resolved by DNS server from vCenter.
If HCX manager cannot connect to this IP address, Service Mesh creation will fail with the above error.

Resolution

This is caused by HCX expected behavior mentioned above.
To resolve this, HCX manager has to be able to connect to ESXi's vmk IP address which is ESXi's DNS resolution IP address from vCenter.

For example, If the result of nslookup from vCenter to ESXi is 192.168.0.1 (vmk0), HCX manager has to be able to connect to 192.168.0.1 (vmk0) while the Service Mesh is being created.