vCenter 6.x/7.x/8.x incorrectly displays the amount of licenses in use by ESXi hosts

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

The purpose of the KB article is to determine if there are stale entries in the VMware Directory Service database, and if so, instructions on how to remove them.

 

Steps can also be used to remove expired trial licenses & stale licenses in general.

Symptoms:

• vCenter Server shows the license assigned key count as negative
• You are unable to assign more hosts to a license key
• The number of hosts in the environment is lower than the capacity of the license

Environment

VMware vSphere 6.x

VMware vSphere 7.x

VMware vSphere 8.x

 

Cause

This issue can occur due to a stale license entry in the VMware Directory Services database.

Resolution

To resolve this issue, determine if there are stale entries in the VMware Directory Service database and remove them.

Note: Ensure that the vCenter has been rebooted before attempting this KB, Additionally offline snapshots of all linked nodes should be taken prior.
 

To identify and remove stale license entries:

 

Note: This process involves modifying the VMware Directory Services database. Ensure to take a backup of the database before proceeding. For more information, see the Backup section in List of recommended topologies for VMware vSphere 6.0

1. Run select * from VPX_ENTITY where TYPE_ID = 1 on the Windows vCenter Server database to display the hosts:

vCenter Server Appliance:

1. Log in to the vCenter Server Appliance.
2. Type shell.set --enabled true and press Enter.
3. Run this command to retrieve the database password:
   
   cat /etc/vmware-vpx/vcdb.properties | grep "password =" | awk '{ print $3 }'

4. Connect to the database by running this command:

/opt/vmware/vpostgres/current/bin/psql -d VCDB vc
 

Note: Enter the password from Step c when prompted.

 

 

Note: in the rare case the above command to enter the postgres CLI does not work, there is a "brute-force"-alternative. Due to the extended rights of the postgres-user, using the above way via the vc user is to be preferred.

 

/opt/vmware/vpostgres/current/bin/psql -U postgres -d VCDB

 

5. Run this query:
   
   SELECT * FROM vpx_entity WHERE type_id = 1;

6. Make a note of the hosts and their associated ID field displayed in the query results.
   
   Note: This process must be repeated for each vCenter Server in the VMware vCenter Single Sign-on domain.
    
7. Download and install JXplorer, available at JXplorer.
8. Open JXplorer and select File > Connect.
9. In the Open LDAP/DSML pane, make these configuration changes:
   1. For host, enter the FQDN of your Platform Services Controller.
   2. The Port should be 389 or 11711.
   3. The protocol is LDAP v3.
   4. Base DN will be dc=vsphere,dc=local.

Note: If you are using a custom vCenter Single Sign-on domain name, you have to replace vsphere and local with the name of your custom domain name. For example, if you chose vsphere.vmware.corp as your vCenter Single Sign-on domain, the value would be: dc=vsphere,dc=vmware,dc=corp

10. The Security Level will be User + Password.
11. The Security User DN will be cn=administrator,cn=users,dc=vsphere,dc=local.
12. The Security Password will be your administrator password for your vCenter Single Sign-on administrator account.
13. Expand Services > LicenseService.
14. Select any AssetEntity_host-ID-UUID values that do not have a corresponding match to the information obtained from the vCenter Server database in step 2.

 

Note: The ID fields from both sources should match. The UUID object is unique to each vCenter Server within the vCenter Single Sign-On.

15. Confirm the assets selected are not currently in use by:
     
16. Select the asset in JXplorer to view these properties of the object:
    
    vmwLicSvcAssetName - Friendly name of the asset. This can be the Fully Qualified Domain Name (FQDN) or IP address of the asset.
    vmwLicSvcAssetScopeID - This is the license associated with the asset.

 

17. Using the vmwLicSvcAssetScopeID value from step 9a, navigate to the corresponding LicenseEntry_vmwLicSvcAssetScopeID entry under Services > LicenseService within JXplorer where vmwLicSvcAssetScopeID is a unique identifier assigned to the license entry.

Review the vmwLicSvcLicenseName and vmwLicSvcLicenseSerialKeys value to confirm the friendly name and key of the license that reports the incorrect amount of licenses in use.

 

18. Remove the affected AssetEntity_host-ID-UUID.
    
    Warning: Do not attempt to remove or modify any objects that do not begin with AssetEntity_host-. Removing or modifying these objects may require a clean reinstall of vCenter and the Platform Services Controller without preserving vCenter Single Sign-on data.
     
    1. Right-click on the entry to remove.
    2. Select Delete.

Additional Information

• Managing licenses on ESXi hosts using the vSphere Web Client
• To find your SSO domain from SSH, use the command :

/usr/lib/vmware-vmafd/bin/vmafd-cli get-domain-name --server-name localhost

Impact/Risks:
Warning: Before using JXplorer to remove stale licenses or make any changes, ensure you have taken proper snapshots of your SSO domain. This means that you must shut down all vCenters or PSCs that are in the SSO domain at the same time, then snapshot them, and power them on again.  If you need to revert to one of these snapshots, shut all the nodes down, and revert all nodes to the snapshot. Failure to perform these steps will lead to replication problems across the PSC databases.