Two VMs are unable to communicate with each other

Products

• VMware NSX

Issue/Introduction

• NSX VM to VM communication issue with higher MTU size [ above 1414 size ]
•  Both VM's are part of different esxi host and two different segments which is mapped to different Tier-1's.
•  Validate the MTU config on the exsi host and edge VM mapped esxi host as well. 

Environment

VMware NSX

Cause

• If there is an inconsistent MTU configuration on the physical network L3 interface which was causing the ping failure for different TEP networks.
• TEP connectivity between host to edge and vice verse is between two different network, so the routing is performed on the physical L3 interface.

Resolution

• To validate the TEP connectivity from edge node TEP to host TEP and TEP gateway IP with 1500 MTU size, execute the below command on the edge node and esxi host.
  
  vmkping -I vmk## -S vxlan -d -s 1572 <destination TEP IP>

ping <destination-ip> source <source-ip> size<> dfbit enable
  
  
•  Perform a packet capture on the problem esxi host uplink and edge vm mapped esxi host to validate the ICMP packets going through the source esxi host and If the same packet is reaching the edge vm mapped esxi host or not.
   
  pktcap-uw --uplink [VMNIC] --capture UplinkSndKernel,UplinkRcvKernel --rcf 'geneve and host <ip>' -o - | tcpdump-uw -enr - 
  
  
  
• Kindly follow this KB article to address common NSX Underlying Infrastructure Connectivity Issues - https://knowledge.broadcom.com/external/article?articleNumber=317210