HCX IX Mobility Agent host randomly gets disconnected from vCenter Server after established Service Mesh
search cancel

HCX IX Mobility Agent host randomly gets disconnected from vCenter Server after established Service Mesh

book

Article ID: 323280

calendar_today

Updated On:

Products

VMware HCX VMware Cloud on AWS

Issue/Introduction

  • After established HCX Service Mesh from HCX Connector, Mobility Agent virtual host which is deployed by HCX Interconnect randomly reports "Host connection and power state" alarm in vCenter Server.
    Alarm 'Host connection and power state' on <Mobility Agent IP> changed from Green to Red
  • The vCenter Server vpxd log (/var/log/vmware/vpxd/vpxd.log) files contain entries similar to:
    YYYY-MM-DDT<TIME>+09:00 info vpxd[7F259F972700] [Originator@6876 sub=HostCnx opID=CheckforMissingHeartbeats-4332c98c] [VpxdHostCnx] No heartbeats received from host; cnx: ####-####-####-####-####, h: host-####, time since last heartbeat: 73361118066ms
    YYYY-MM-DDT<TIME>+09:00 info vpxd[7F259F972700] [Originator@6876 sub=HostCnx opID=CheckforMissingHeartbeats-4332c98c] [VpxdHostCnx] Marking the connection alive to false: ####-####-####-####-####
    YYYY-MM-DDT<TIME>+09:00 info vpxd[7F259F972700] [Originator@6876 sub=InvtHostCnx opID=CheckforMissingHeartbeats-4332c98c] [VpxdInvtHost] Got lost connection callback for host-####
    YYYY-MM-DDT<TIME>+09:00 warning vpxd[7F259EDDB700] [Originator@6876 sub=InvtHostCnx opID=HostSync-host-####-b77f594] Connection not alive due to missing heartbeats; [vim.HostSystem:host-####,172.##.##.##], cnx: ####-####-####-####-####
    YYYY-MM-DDT<TIME>+09:00 info vpxd[7F259EDDB700] [Originator@6876 sub=InvtHostCnx opID=HostSync-host-####-b77f594] Succeeded restoring heartbeat; [vim.HostSystem:host-####,<IP>]
    YYYY-MM-DDT<TIME>+09:00 info vpxd[7F259EDDB700] [Originator@6876 sub=MoHost opID=HostSync-host-####-b77f594] [HostMo] host connection state changed to [CONNECTED] for host-####




Cause

The Mobility Agent is a virtual host which is deployed by HCX Interconnect.
This issue will usually occur if vCenter Server is unable to communicate with Mobility Agent host via TCP443 and TCP/UDP 902 , same as typical ESXi hosts.

Resolution

Please check and reconfigure the firewall to allow access on the appropriate ports and make sure TCP443 and TCP/UDP 902 are not blocked between HCX Mobility Agent host and vCenter Server.
After validating the required ports, if the issue persists, you may re-deploy the HCX Interconnect.

Additional Information