HCX Service Mesh Diagnostics Reports TCP 902 Failure to Empty Destination IP Address

Products

• VMware HCX

Issue/Introduction

Symptoms:

• When running HCX Service Mesh Diagnostics, the test reports TCP port 902 connection failures from both the HCX Manager IP address and HCX IX management IP address to an empty destination IP.  An example of this result is below:
  
• You may also observe following log entries under /common/logs/admin/app.log

  {"type":"REACHABILITY_TCP_CONNECT","source":"<IX-appliance-IP_address>","destination":"","sourcePort":0,"destPort":902,"destType":"Deployment_HostSystem","protocol":"TCP","status":"FAILURE","error":{"output":"dial tcp :902: connect: connection refused","message":"Probable issues: Port closed on target, Service not running on Target."},"timestamp":1769685513}
  
  <Time-stamp> [InterconnectService_SvcThread-21159, J:73dcd2d2, , TxId: <Transaction-ID>] WARN  c.v.v.h.s.i.c.ComputeProfileDiagnosticJob- Found a duplicate probe for [{"ports":[902],"commands":["TCP_CONNECT"],"protocol":"TCP","serviceName":"OVF Import"},{"ports":[80],"commands":["HTTP_CONNECT","GET","/sdk/vimService.wsdl"],"protocol":"TCP","serviceName":"OVF Import"},{"ports":[443],"commands":["HTTPS_CONNECT","GET","/sdk/vimService.wsdl"],"protocol":"TCP","serviceName":"OVF Import"}], skipping it

• The ESXi hosts selected in the Deployment and / or Service Selection of the respective Compute Profile for this given Service Mesh contain ESXi hosts that have an IPv6 vmkernel adapter address that is designated for management.  An IPv6 vmkernel adapter that is designated for management is common in VxRail environments.
• The Destination address in the Diagnostics output should contain the IP/vmkernel adapter of the ESXi host designated with the Management Service of the host(s) where the VM(s) are currently running.
  

Environment

VMware HCX

Cause

HCX Service Mesh Diagnostics performs a network port probe to help users identify connectivity issues between HCX appliances and ESXi hosts within the Service and Deployment selection of a Compute Profile.  Destinations to test are determined by the services enabled on ESXi host vmkernel adapters.  HCX is not compatible with IPv6, yet will probe to all management enabled vmkernel adapters.  This results in a reported failure to an empty destination IP address.

Resolution

There is currently no resolution to this behavior.

Workaround: Remove the management designation from all vmkernel adapters that do not have an IPv4 address.  This workaround is often not desirable, nor recommended if the IPv6 vmkernel adapter is in use.

Additional Information

Impact/Risks: There is no impact associated with this behavior.  This can be safely ignored.