NSX Host Transport node Central Control Plane connectivity down due to running an older NSX version

Products

• VMware NSX

Issue/Introduction

• VMware NSX or NSX-T Data Center is installed.
• The ESXi Host transport node is running an older version to the rest of the environment.
• Log lines similar to the below are encountered on the NSX Manager in var/log/syslog

  NSX 1591 - [nsx@6876 comp="nsx-controller" errorCode="CCP0310030" level="ERROR" subcomp="handshake-server"] Unsupported Versions reported by TN ########-####-####-####-############: [major: 3#012minor: 1#012, major: 3#012minor: 0#012, major: 2#012minor: 5#012]

• The connectivity to the controllers will be down with port 1235 on the ESXi to the NSX Manager Nodes in TIME_WAIT:

  esxcli network ip connection list | grep -iE "1235.*time_wait"
  tcp         0       0  <TN node IP>:<Port>              <nsx_mgr_1>:1235    TIME_WAIT           0
  tcp         0       0  <TN node IP>:<Port>              <nsx_mgr_2>:1235    TIME_WAIT           0
  tcp         0       0  <TN node IP>:<Port>              <nsx_mgr_3>:1235    TIME_WAIT           0

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware NSX 4.x
VMware NSX-T Data Center 3.x

Cause

This behavior is expected when the host VIB is a lower version to the NSX Manager VIB

Resolution

This is a condition that may occur in a VMware NSX environment.

 

Workaround

• Upgrade the host VIBs at the ESXi level
  1. Place the host in maintenance mode.
  2. Download the NSX VIB from the NSX Manager using the steps outlined in KB 371846.
  3. Execute the command to install the VIBs outlined in the Installation guide. Example:

     eg: esxcli software vib install -d /tmp/nsx-lcp-<release>.zip

• Uninstalling and reinstalling NSX along with reimaging the ESXi host should also resolve this issue, but care should be given should the environments be VCF and if VSAN is in use.