HCX - Service Mesh diagnostics test returns 2 failed probes

Products

• VMware HCX

Issue/Introduction

Identify a known issue with respect to HCX SM diagnostic test.

Symptoms:
HCX Service Mesh (SM) diagnostic test performed through UI returns 2 failed probes as shown below:

Navigate to "HCX Manager UI -> Interconnect -> Service Mesh ->  More  -> Run Diagnostics"

 

Note: In HCX Manager Service Mesh wizard, all diagnostic tests will be passed (green check mark) yet failed probes will be reported.

Below errors will be seen in HCX app-engine logs during diagnostics tests:

/common/logs/admin/app.log

INFO  c.v.v.h.s.i.EndpointDiagnosticsJob- Diagnostics Response from hcmProbe: 

{"type":"REACHABILITY_PING","source":"<source ip address>","destination":"<destination ip address>","sourcePort":0,"destPort":0,"protocol":"ARP","data":{"maxLatency":"","minLatency":"","avgLatency":"","probeCount":0,"probesLost":0,"probes":null},"destType":"STATIC-ROUTE-GATEWAY","status":"FAILURE","timestamp":1706733769,"error":{"output":"/usr/sbin/arping -c 2 -w 1 -I eth0 <destination ip address>: exit status 1","message":""},"commandOutput":"/usr/sbin/arping -c 2 -w 1 -I eth0 <destination ip address>\nARPING <destination ip address> from <source ip address> eth0\nUnicast reply from <destination ip address> [mac:address]  6.271ms\nSent 2 probes (1 broadcast(s))\nReceived 1 response(s)\n"}]

Environment

HCX 4.8

Cause

In our current implementations of diagnostics test, the timeout value configured for ARPING command is a bit aggressive and may not be efficient to discover hosts within stipulated amount of time. As a result, it may return arp probe failure.

Note: arping is a tool for probing hosts in a given network. It operates at the data link layer and uses the Address Resolution Protocol (ARP) request to the destination host and wait for subsequent ARP reply.

Resolution

This issue is fixed in HCX 4.9 or later versions.

Additional Information

This issue is also mentioned in the release notes for HCX 4.9 version. 

 

Impact/Risks:

• The HCX Service Mesh diagnostic probe failure causing a false positive alarm and should be considered as cosmetic.
• This issue is only applicable to HCX version 4.8.x.
• Migration and Network Extension services will remain unaffected.
• Diagnostics test will be completed successfully without any functional impact.