Enabling vLCM fails on NSX enabled host fails with error "Error occurred while checking whether NSX-T Data Center is enabled on this standalone host" .

Products

• VMware NSX

Issue/Introduction

• Attempting to enable image-based updates fails. The error message presented in the GUI is "Error occurred while checking whether NSX-T Data Center is enabled on this standalone host". This appears during the "Check host's eligibility to be managed with a single image" step.
• Additionally, this error message can also be observed when trying to convert a cluster to an image-based vLCM cluster.
• CA-Signed certificates are in use within the environment.

Environment

VMware NSX
VMware NSX-T Data Center

Cause

If the NSX Manager's API certificate is CA-signed, the NSX Manager does a reverse-DNS lookup on the NSX Manager's IP address, and a forward lookup of the NSX Manager's hostname to vCenter.  
The NSX Manager requires that the same hostname matches either the CN (common name) or a SAN entry in the NSX Manager's API certificate. Without this requirement install the upgrading of NSX components will fail.   NSX components (eg other NSX Managers or NSX Edges) act as clients for the NSX Manager's REST API interface. The hostname check of the process will fail the client-side certificate validation.

Resolution

If the NSX Manager's certificate is CA-signed, then the DNS server must provide for forward and reverse lookups of the Manager's IP address and Manager's hostname.  The NSX Manager's hostname must be included in the NSX Manager's API certificate, either as a CN (common name), or as a SAN (subject alternative name) entry.  The SAN entry may contain a wild-card, eg, " * ".    For instance,  *example.com would match the following: mgr1.example.com, mgr2.example.com and mgr3.example.com .

This can be confirmed by carrying out the following steps to verify the required elements.


Verify vCenter MOB extension list.

Validated this against what the vCenter NSX T extension has recorded by checking the below:

1. Open vCenter MOB URL using https://<vCenter Name/IP address>/mob
2. Give sso admin credentials for authentication.
3. Click on "content"
4. Search for "Extension Manager" and click on it.
5. Click on "more" of Extension list to list all the extensions
6. Look for "extensionList["com.vmware.nsx.management.nsxt"]" and click on it.
7. Click "server"
8. Look for "serverThumbprint"

This should list all of the NSX managers unique FQDN as well as the VIP along with the thumbprint matching the certificate that is assigned within NSX-T to that respective node that has been retrieved by vCenter. If the expected certificate thumbprint is showing to the correct FQDN in the NSXT extension then the certificate may need validating. If the FQDN is incorrect or repeated, DNS is likely incorrect for the IP/FQDNs in use and should be validated.


Verify DNS

Verify the DNS mapping for all manager node IPs, including the VIP, by using the "dig" command as outlined above. This will help confirm the accuracy of the DNS configuration.
From the vCenter:

1. Run "dig -x IP_ADDRESS +short" to perform a reverse DNS lookup on each of the Manager's IP address'.
2. Run "dig FQDN +short" and verify that the result matches the IP from the previous step. If it does not, the DNS server is misconfigured and needs to be configured correctly.


Verify Certificates

The CA-signed certificate should also include the FQDN, either in the CN (Common Name) or SAN (Subject Alternative Name) fields. This can be checked by using the below commands on each manager node, if the certificate does not contain the FQDN then update the certificate for the node accordingly.

  1. Log into the manager as root.   If already logged in as admin use "st en" This will change the user level to root after entering the root password.
  2. Use the following command to retrieve the node’s certificate:
   echo | openssl s_client -showcerts -verify 5 -connect localhost:443 2>/dev/null> certificate.pem
   Output is saved to the file called  `certificate.pem`.

   3. To extract the expected FQDN from the certificate, run:
   openssl x509 -in <certificate.pem> -text -noout |  grep -e"CN" -e"DNS"