Host transport node upgrade failed with error: "Unexpected error while upgrading upgrade unit. Host https://<host_ip>/sdk not reachable. java.net. NoRouteToHostException: No route to host (Host unreachable)"

Products

• VMware NSX

Issue/Introduction

• Host transport node upgrade failed with below error.
  • Error "Unexpected error while upgrading upgrade unit. Host https://<host_ip>/sdk not reachable. java.net. NoRouteToHostException: No route to host (Host unreachable)"

• This issue may report on one or more host transport nodes where there are 2 VMkernel adapters configured to use same management service and one of the IP is not reachable.

Environment

VMware NSX

Cause

If there are 2 management VMkernel adapters with different IPs, NSX manager selects any one of the IP to connect.

• The failure stems from a dual-VMkernel "management service" configuration. When management services are enabled on two adapters and one IP is unreachable, it disrupts the communication path between the ESXi host and the NSX Manager when the communication goes through the unreachable IP.
  OR
• Host management IP is not reachable over the network from NSX manager.

Resolution

• Ensure that stable network connectivity is established with the ESXi host with specific IP address reported in the NSX Manager upgrade error.
  
  
  • Place the host into maintenance mode.
  • Check both the vmkernel adapters reachability over the network. Refer to the KB Testing VMkernel network connectivity with the vmkping command.
  • If upgrade needs to be prioritize without fixing the IP reachability issue, temporarily disable management service on unreachable vmkernel adapter.
  • Retry the upgrade from NSX manager.
  • Once upgrade is completed, Re-enable the management service on problematic VMkernel adapter and further investigate the IP reachability issue.
  • Check if there are multiple vmkernel adapters configured for management network on the ESXi host.

Additional Information

Reference KB for the communication issue between the host and NSX manager (No route to host): Unable to install NSX-T packages on a specific ESXi host