"Licenses could not be synchronized with the vCenter systems", unable to assign the vCenter license from VCF Operations

Products

• VMware vCenter Server
• VMware Cloud Foundation
• VCF Operations

Issue/Introduction

• Unable to license the vCenter from VCF Operations, Business Services Console shows the licenses as Active with '0' cores used.
  
  
  
  
• From the License Management page on VCF Operations, the Assign Primary License page is properly showing the license as below:
  
  
  
  
  
  
• Assign license task fails with error message below:
  
  Licenses could not be synchronized with the vCenter systems.
Licenses could not be assigned to the selected vCenter systems.
  
  
  
  
• License plugin logs on VCF Operations appliance shows CONNECTION_FAILED errors as below:
  
  /storage/log/vcops/log/vcf-licensing-plugin.log

YYYY-MM-DDTHH:MM:SS.###Z NOTICE vcf-licensing-plugin 5491 [ops@4413 threadId="140316" threadName="pool-41-thread-1110"] [com.vmware.vrops.licensing.vcf.A.C.A] - Executed assign entitlement task on adapter instance completed with result [AssignEntitlementStatus(allocationId='########-####-####-####-############', vcServerGuid=########-####-####-####-############, vcHostname='<vcenter_fqdn>', vcAdapterName='<vcenter_name>', vcfAdapterName='null', status=CONNECTION_FAILED, usageInfo-[], entitlementUsageInfo-[])]
YYYY-MM-DDTHH:MM:SS.###Z NOTICE vcf-licensing-plugin 5491 [ops@4413 threadId="140316" threadName="pool-41-thread-1110"] [com.vmware.vrops.licensing.vcf.A.C.E] - Storing completed assign entitlement task for task id ########-####-####-####-############: [VcAssignEntitlementStatus(allocationId=########-####-####-####-############, entitlementName=<LicenseName>, vcId=########-####-####-####-############, vcHostname=<vcenter_fqdn>, vcAdapterName=<vcenter_name>,vcfAdapterName=null, voStatus=FAILED, reason=CONNECTION_FAILED)

Environment

• VVF 9.x
• VCF 9.x

Cause

• This issue is observed when VCF Operations appliance is unable to communicate with the vCenter Server due to DNS resolution failures.
• DNS name resolution using 'nslookup' command on VCF Operations appliance can confirm the status. If DNS is not configured properly, nslookup fails with error below:
  

  root@<vcf-ops-hostname> [/]# nslookup vcenter_hostname.example.com
;; communications error to <VCF Ops IP>#53: connection refused
;: communications error to <VCF Ops IP>#53: connection refused
;: communications error to <VCF Ops IP>#53: connection refused
;; no servers could be reached

  root@<vcf-ops-hostname> [/]#

  Note: The vCenter FQDN can be taken from the log snippets from /storage/log/vcops/log/vcf-licensing-plugin.log

Resolution

Configure proper DNS servers on VCF Operations appliance by following the steps below:

1. Shutdown VCF Operations Appliance.
2. Login to vSphere Client and locate the VCF Operations VM.
3. From the VM summary page, click Configure.
4. Click Settings > vApp Options.
5. Click the Radio button next to the setting with Label Domain Name Servers and click Set Value.
   
   
   
   
6. Edit the Domain Name Servers IP address(es) in the Property Value field. Use a comma to separate 2 IP addresses.
7. Click OK to close the Set value dialog.
8. Power On the Operations appliance VM.
9. Verify the DNS resolution using 'nslookup' command and retry the license assignment.

Additional Information

Modifying the DNS settings of VMware Aria Operations.