HCX-IX/NE tunnels down

Products

• VMware HCX

Issue/Introduction

• HCX IX/NE tunnels shows down as below in UI
  
  
  
  
• Service mesh diagnostic test shows failed probes for UDP port 4500
  
  
  
  
• Service Mesh diagnostic test shows failed probes between the Source & Target IX/NE appliances
  
  
  
  
• Ping test between the Source and Target IX/NE appliances is failing.

• UDP port 4500 validation is also failing. You can verify port connectivity using the following command:

  curl -kv <IX/NE-IP>:4500

• To capture traffic on the IX/NE appliances, use the command:

   

  tcpdump -i vNic_0 -S -nn | grep 4500

  (Replace vNic_0 with the actual uplink interface name)

• Packet captures on the IX/NE uplinks indicate unidirectional UDP traffic, suggesting possible firewall or routing issues.

Environment

VMware HCX

Cause

The tunnels may be down due to the following reasons:

• A network connectivity issue between the Source and Target IX/NE appliances.

• Firewall rules allowing only unidirectional UDP traffic between the Source and Target Interconnect (IX) appliances.

Resolution

 

• Ensure there is network connectivity between the Source and Target IX/NE appliances.

• Verify that UDP port 4500 is open and accessible between the Source and Target IX/NE appliances.

• Review packet captures between the IX/NE appliances to confirm that traffic is bi-directional, not just one-way.

 

Additional Information

HCX Health Check and Best Practices

Note :- To perform tests via ping, curl and tcpdump, follow the steps below:-

• • Log into the HCX manager via ssh as user admin
  • Enter into the Central Command Line mode by running the ccli
  • Run the command list to list all appliances IPs and connection status
  • Note down the corresponding IX/NE appliance name and Id number that has tunnel issue
  • To access the shell of the specific IX/NE appliance, run the command ssh
  • Post entering the shell mode, ping, curl and tcpdump commands can be executed.