Error: "Licenses could not be assigned to vCenters", seen during license assignment on VCF 9.1
search cancel

Error: "Licenses could not be assigned to vCenters", seen during license assignment on VCF 9.1

book

Article ID: 424533

calendar_today

Updated On:

Products

VCF Operations VMware vCenter Server VMware Cloud Foundation

Issue/Introduction

  • License assignment to a selected vCenter in VCF Operations 9.1 fails with an error message "failed assignments" and "Licenses could not be assigned to the selected vCenter".



  • Selecting the View Details tab for the error message shows the message below:

    This license could not be assigned to the selected vCenter system. Check the network connection between the license server and the selected vCenter system and try again.


    This license could not be assigned to the selected vCenter system. vCenter is not able to resolve the hostname of license server.
  • An error message may show on the licenses page saying: “Licenses could not be synchronized with vCenters.” or  “Add-on licenses could not be synchronized with vCenters.”



  • Navigating to VCF Operations UI > Manage > Licensing > Licenses and Registration > VCF License shows Assignable as NO with error message as "License needs to be updated and assigned to the license server"

  • /storage/log/vcops/log/collector.log on VCF Operations may show the following error:

    com.vmware.adapter3.vmwareinfrahealth.helpers.vcenter.collector.VCenterAvailabilityMetricsCollector.addNTPServerAndDriftMetric] [NTP DRIFT] NTP drift is not performed cause this setup vCenter-<vCenterFQDN> as its vROPs is not configured with NTP server.

Environment

  • VMware Cloud Foundation (VCF) 9.1
  • VMware vSphere Foundation (VVF) 9.1
  • vCenter Server 9.x
  • VCF Operations 9.1

Cause

This issue occurs due to network connectivity failures or time drift disrupting communication between VCF Operations, the vCenter Server, and the license server.

Alternatively, this issue occurs when a vCenter Server instance upgraded from version 5.x retains an uppercase instanceUuid (vcGuid) inside the configuration files /etc/vmware-vpx/instance.cfg and /etc/vmware-vpx/firstboot/vpxd-service-spec.prop. Uppercase identifiers prevent successful authentication requests to the license server.

Resolution

Follow the sections below to verify connectivity, synchronize time, and resolve configuration blocks across the environment.

Verify License Server deployment and Power Status

  1. Navigate to Inventory in the vSphere Client.

  2. Select the cluster where the license server virtual machine (VM) runs.

  3. Verify if the license server VM is deployed:

    • If not deployed: Deploy the license server VM following the official post-upgrade documentation for VCF 9.1 before proceeding.

    • If deployed: Select the license server VM.

  4. Verify that the Power Status shows Powered On.

  5. If the VM is powered off, select Actions > Power On.

Synchronize Component Time (NTP)

Time drift exceeding 30 seconds between the license server, vCenter Server, and VCF Operations causes authentication and synchronization failures.

  1. Open SSH sessions to the ESXi host running the license server, the vCenter Server, and the VCF Operations appliance.

  2. Run the following command in quick succession on each component to check the current UTC time:

    date -u +"%Y-%m-%dT%H:%M:%SZ"

  3. If timestamps differ by more than 30 seconds, configure an identical NTP server across all components:

Verify vCenter Server Connectivity to the License Server

  1. Check the status of current license assignments:

    • If a license was previously assigned, verify if the vCenter Server user interface displays an alert banner indicating the connection to the license server has been lost.
    • If a license has not yet been assigned, proceed to validate direct network layer communication from the command line.

  2. Access the vCenter Server appliance via SSH.

  3. Validate TCP port 443 connectivity to the license server by running:

    curl -v -k https://<license-server-IP>

    Alternatively, run:
    openssl s_client -connect <license-server-IP>:443

  4. If the connection fails or blocks, add an explicit firewall rule to allow TCP port 443 traffic from the vCenter Server IP address to the license server IP address.

  5. Verify the vApp network settings if the license server IP address or gateway requires correction. In the vSphere Client, power off the license server VM, navigate to Configure > vApp Options, update the fields, and power on the VM.

  6. Test forward and reverse DNS resolution from the vCenter Server using nslookup:

    nslookup <license-server-hostname> <vCenter-DNS-IP>
    nslookup <license-server-IPAddress> <vCenter-DNS-IP>

    • To identify the vCenter Server DNS IP address: Log in to the vCenter Server Appliance Management Interface (VAMI) at https://<vcenter_IP_or_FQDN>:5480. Navigate to Networking, and under Networking Settings, click Edit > Next, then expand Hostname and DNS to locate the configured DNS server entries.

    • To identify the license server hostname and IP address: Locate the license server VM in the vCenter Inventory, launch the Remote Console, and retrieve the values listed under Hostname and IP Address.

  7. If the lookups fail, create an A record for the license server hostname in the primary DNS server utilized by vCenter Server.

Verify License Server FQDN Resolution 

  1. Access the License Server VM console via the vSphere Client.
  2. Observe the FQDN displayed on the login screen.
  3. Check for anomalies, such as a double-appended domain suffix (e.g., ####.domain.net.domain).
  4. If the FQDN is incorrect, navigate to the VM vApp Options settings.
  5. Update the Hostname to the short name only (e.g., ####) and the Domain Name to the correct domain (e.g., ####).
  6. Power off and power on the VM to apply the changes.
  7. Verify the console now displays the correct FQDN matching the environment's DNS records.

Verify License Server Connectivity to VCF Operations

  1. In VCF Operations, navigate to Manage > Licensing > Licenses & Registration.

  2. Under Registration and License Server Status, click Manage > License Servers to inspect the Connectivity to VCF Operations column.

    • Note: The status may falsely display as connected in this user interface if the Licensing Server vApp was deployed with a valid IP address but possesses missing or invalid DNS records.

  3. If the status shows Not Connected or if the server does not appear after enrollment, open an SSH session to a host on the same management subnet as the license server and test port 443 access to VCF Operations:

    curl -v -k https://<VCF-Operations-IP>

  4. If blocked, configure an allow rule on the network firewall for TCP port 443 from the license server subnet to the VCF Operations IP address.

  5. Verify DNS resolution for the VCF Operations FQDN from the license server's designated DNS server:

    nslookup <VCF-Operations-hostname> <license-server-DNS-IP>

  6. If the returned IP address is incorrect or returns NXDOMAIN, update the A record in the DNS server, or correct the vApp DNS options by powering off the license server VM and adjusting Configure > vApp Options > Domain Name Servers.

Verify VCF Operations Connectivity to vCenter Server

  1. In VCF Operations, navigate to Operate > Inventory, select the specific vCenter targeted for license assignment, and select the Alerts tab.

    • Verify if an alert reading Connectivity to the vCenter is affected is present, confirming an active communication block via the user interface.



  2. Access the VCF Operations appliance via SSH.

  3. Test TCP port 443 outbound access to the vCenter Server by running:

    curl -v -k https://<vCenter-IP>

  4. If blocked, create a network firewall rule allowing TCP port 443 from the VCF Operations IP address to the vCenter Server IP address.

  5. Verify hostname resolution using nslookup:

    nslookup <vCenter-hostname> <VCF-Operations-DNS-IP>

  6. If resolution fails, add the appropriate A record to the DNS server configured on the VCF Operations instance.

Correct Uppercase vCenter Server Instance UUID

  1. Launch the vCenter Server appliance shell via SSH.

  2. Check the casing of the unique identifier by running:

    grep instanceUuid /etc/vmware-vpx/instance.cfg

    Example output:
    instanceUuid=DBE#####-1##B-##BC-CA##-##CDC#####

  3. If the instanceUuid contains uppercase letters, convert the identifier to lowercase. For detailed steps, see the resolution section of KB - Registering VASA provider fails with error "A problem was encountered while registering the provider"

Check License Server Health and Expand Storage

  1. In VCF Operations, navigate to Operate > Dashboards > All > License Server Health.

  2. Select the target license server to review CPU, memory, and disk utilization.

  3. If CPU or memory utilization permanently exceeds 90%, reboot the appliance from the vSphere Client by navigating to the VM and selecting Actions > Power > Reset.

  4. If a disk partition (root or data disk) is full, expand the virtual hardware storage allocation:

    • Gracefully shut down the guest operating system of the license server VM (Actions > Power > Shut Down Guest OS).
    • Right-click the VM and select Edit Settings.
    • Expand the full Hard Disk entry, increase the provisioned size, and click OK.
    • Power on the virtual machine (Actions > Power > Power On). The appliance automatically resizes the partition during the boot process.

Additional Information

Connectivity diagram between License Server and other components:

The account used by the vCenter Adapter has to be a member of the LicenseService.Administrators Single Sign-On group.
See KB article, Error "License key(s) could not be added to vCenter system(s)" When Adding vCenter License in Aria Operations

 

License server is expected to get time from the ESXi hosting it. However it may incorrectly attempt to connect to an external NTP server to update its internal clock per following release note:

https://techdocs.broadcom.com/us/en/vmware-cis/vcf/vcf-9-0-and-later/9-1/release-notes/patch-releases-9-1-0-x/vcf-operations/9-1-0-0400/vcfoperations-9-1-0-0400-release-notes.html

  • License server attempts to connect to external Network Time Protocol (NTP) servers for time synchronization
    The license server incorrectly attempts to connect to an external NTP server to update its internal clock.
    Starting with VCF Operations 9.1.0.0400, time synchronization for the license server is restricted exclusively to the host system on which the license server virtual machine runs.

     

    Powered by Wolken Software