ESXi update fails with error 'Failed to run health checks for NSX-T' for vLCM cluster.

Products

• VMware NSX

Issue/Introduction

• During ESXi update, vCenter is generating following error for hosts in vLCM cluster.

Remediation of cluster failed

 Failed to run health checks for NSX-T on '<cluster-name>'.
 Health Check for '<cluster-name>' failed
<host-name> - Skipped remediation for this host

• Cluster may not be prepared for NSX.

 

Environment

• VMware NSX-T Data Center
• VMware NSX

Cause

vCenter logs show that it is trying to run health check API on a NSX orchestrator node and is failing to reach the node.

Log location -
/var/log/vmware/vmware-updatemgr/vum-server

info vmware-vum-server[938377] [Originator@6876 sub=EHP opID=77b2####-####-####-####-07b0ba09####] Acquiring NSX-T orchestrator node through /node/services/install-upgrade API.
info vmware-vum-server[938377] [Originator@6876 sub=EHP opID=77b2####-####-####-####-07b0ba09####] Successfully found the NSX-T orchestrator node : https://<NSX-MANAGER>:443
info vmware-vum-server[938377] [Originator@6876 sub=EHP opID=77b2####-####-####-####-07b0ba09####] Calling NSX-T API /api/v1/vlcm/esx/health/cluster/perspectives/ready-for-apply/status?action=check (/external-tp/http1/<NSX-MANAGER>/443/8599a1c5####1b0facaf7217ebef1190334b149cd46402219820fb6d97d1####/api/v1/vlcm/esx/health/cluster/perspectives/ready-for-apply/status?action=check).
warning vmware-vum-server[938377] [Originator@6876 sub=EHP opID=77b2####-####-####-####-07b0ba09####] Retrying on next NSX-T node due to HTTP 503.
error vmware-vum-server[938377] [Originator@6876 sub=EHP opID=7b2####-####-####-####-07b0ba09####] No reachable NSX-T node found.

This is because orchestrator node is not accessible from the vCenter Server.

Resolution

• Ensure vCenter is able to connect to NSX orchestrator node on port 443
• Ensure proper name resolution is in place for all NSX manager nodes. See Missing DNS entry for manager FQDN alarm in NSX Manager for more information
• In case orchestrator node is down, configure any other node as orchestrator using NSX manager CLI. 
  
  nsx-manager> set repository-ip
  
  Validate new node has install-upgrade service enabled and running using NSX manager CLI.
  
nsx-manager> get service install-upgrade