Host Transport Nodes in Install Failed state with error "Failed to send HostConfig RPC to MPA"

Products

• VMware NSX

Issue/Introduction

• ESXi hosts are in "Failed/Host Disconnected" status with the error below:

  "Host configuration: Failed to send the HostConfig message.
  [TN=TransportNode/<uuid>]. Reason: Failed to send HostConfig RPC to MPA TN:<uuid>. Error: Unable to reach client <tn-uuid>, application SwitchingVertical"

• Using the Resolve action to reconfigure NSX for the TN keeps failing with the same error.
• On the affected ESXi host(s), the following commands report no disconnection or connectivity problems with the NSX managers and controllers.

  nsxcli -c get managers
  nsxcli -c get controllers

• On the affected ESXi TN(s), /var/run/log/nsx-syslog.log indicates timeouts to the localhost address 127.0.0.1.
• vMotion operations for VMs connected to NSX segments consistently fail.

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware NSX 4.x
VMware NSX-T Data Center 3.x
VCF 9

Cause

While the transport node is connecting correctly to NSX CCP there is an internal failure to forward the configuration commands because nsx-proxy cannot communicate with another internal service such as nsx-opsagent

Resolution

This is a condition that may occur in a VMware NSX environment.

 

Restart the nsx-opsagent and nsx-proxy services from CLI on the affected ESXi TN(s):

•  Log in to the ESXi host CLI via SSH.
  • Restart the services:

    /etc/init.d/nsx-opsagent restart
    /etc/init.d/nsx-proxy restart

  • Verify the service status:

    /etc/init.d/nsx-opsagent status
    /etc/init.d/nsx-proxy status

•  If restarting above services did not resolve the issue, follow resolution provided in this KB ESXi hosts are in "Failed" status with errors "Failed to send the HostConfig message"

 

Note: If restarting the services did not help, you can also perform a reboot of the Transport Node.