NSX Appliances Cluster shows up as UNAVAILABLE in NSX Manager UI

Products

• VMware NSX

Issue/Introduction

• NSX Manager Appliances are not showing up on the NSX UI - Cluster shows as UNAVAILABLE

• No impact observed on data-path
• Following log lines in var/log/proton/nsxapi.log file indicate issues with DNS:
  
  

Example 1 - DNS takes long time to respond

12:49:16.852Z INFO http-nio-127.0.0.1-7440-exec-6 Utils 139026 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="########-656a-4ebd-b393-############" subcomp="manager" username="admin"] Executing script: /opt/vmware/nsx-common/python/nsx_utils/ip_utils.py -d4 ##-nsx-manager.systems.##.####

12:49:32.393Z INFO http-nio-127.0.0.1-7440-exec-6 Utils 139026 - [nsx@6876 comp="nsx-manager" level="INFO" reqId="########-656a-4ebd-b393-############" subcomp="manager" username="admin"] getIpv4FromFqdn(): script returned Ipv4: ##.#.#.##

We can observe above that ip_utils.py took 15 seconds to respond.

Example 2 - DNS does not respond

12:18:44.722Z  INFO EventSource-1-2 Utils 4933 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] getIpv4FromFqdn() invoked with FQDN ;; communications error to ##.##.##.##53: timed out

 

Environment

VMware NSX

Cause

Problems with DNS configuration can lead to this issue. For example, DNS configuration is pointing to a server that does not exist or is unavailable. 

Resolution

Access to the NSX-T Manager command line with admin user

Get the list of IP addresses for the existing DNS server using the following command:
get name-servers

Update the new DNS server IP address using the following command:
set name-servers <new-dns-server>

Delete the old DNS server in case it is not in use by using the following command:
del name-server <new-dns-server>

Name servers are added at the bottom of the list. Therefore to re-order them higher priority ones must be deleted and re-added.

Note: This configuration must be applied to all NSX-T Manager nodes.

 

Additional Information

If this issue occurs even while a secondary DNS is configured then please see NSX management cluster DNS issues observed while secondary DNS is functional and present.

If you believe you have encountered this issue and need assistance in diagnosing the root cause, please open a support case with Broadcom Support and refer to this KB article.

For more information, see Creating and managing Broadcom support cases.