Possible duplicate entries in the Lookup Service Registration Database detected.

Products

VMware vCenter Server VMware vCenter Server 8.0

Issue/Introduction

This article provides information to remove the stale service registrations from vCenter server using SSH session.
vCenter server reports error: "Possible duplicate entries in the Lookup Service Registration Database detected"
Manual method to remove stale service registrations from vCenter Server
Steps in this KB article is applicable for the duplicate service registrations with the same Service Type, Node ID, Owner ID, Site ID and version

Environment

VMware vCenter server 7.x

VMware vCenter server 8.x

Cause

Improper decommissioning of external PSC or vCenter in Enhanced Linked Mode (ELM).
Another vCenter Server in the Single Sign-On (SSO) domain has restarted or is not fully available after restart.
During the re-installation of the vCenter Server, it is possible to have the same vCenter Server registered more than once to the Single Sign-On (SSO).
With a previous install of vCenter Server, SSL certificates were not overwritten or appropriately removed during an upgrade or re-installation

Resolution

Steps to check stale registrations on vCenter Server:

List of the services registered within the vCenter:
- /usr/lib/vmware-lookupsvc/tools/lstool.py list --url https://localhost/lookupservice/sdk --no-check-cert > /tmp/psc.txt
Validate the number of service registrations for each vCenter:
- less /tmp/psc.txt | grep -i "Service Type: " | sort | uniq -c
- For an environment with two vCenters in Enhanced Linked Mode (ELM), there would be two service registrations for each service. If there are more than two, then these can be potentially stale ones.
Identify a valid registration against stale registration using /etc/vmware/install-defaults/vmdir.ldu-guid to compare Node ID from above file (/tmp/psc.txt):
- - For Example:
    - The below service registration belongs to the mentioned Node ID.
      Name: ServiceName
      Description: ServiceDescription
      Service Product: com.vmware.cis
      Service Type: ServiceType
      Service ID: 1f10XX8f-438d-XXb-90f8-47e7d6XXa94f
      Site ID: default-first-site
      Node ID: b2b7XX1e-5XX9-4a55-88XX-699fXXeb1d6f
      Owner ID: [email protected]
      Version: 1.0
      Endpoints:
      Type: com.vmware.cis.data.provider
      Protocol: vmomi
      URL: https://vcsa.example.org:443/sdk
      SSL trust:
      Endpoint Attributes:
      cis.common.ep.localurl: http://localhost:11080/sdk
      Attributes:
      com.vmware.cis.cm.HostId: 603b3XX1-fXXb-4XX2-aXX5-81517XX8f47a
  - The Node ID should match the entry in /etc/vmware/install-defaults/vmdir.ldu-guid.

Scenario 1: When the Node ID is same between service registrations which needs to be removed.

For Example:
Below 2 service registration have the same Node ID, Service Type, Owner ID.

Name: ServiceName
Description: ServiceDescription
Service Product: com.vmware.cis
Service Type: ServiceType
Service ID: 1f10XX8f-438d-XXb-90f8-XXXd6XXa94f
Site ID: default-first-site
Node ID: b2b7XX1e-5XX9-4a55-88XX-699fXXeb1d6f
Owner ID: vpxd-extension@sso_domain

Name: ServiceName
Description: ServiceDescription
Service Product: com.vmware.cis
Service Type: ServiceType
Service ID: 789XXXXfg-64e3-XXb-32u9-XXX862XXX
Site ID: default-first-site
Node ID: b2b7XX1e-5XX9-4a55-88XX-699fXXeb1d6f
Owner ID: vpxd-extension@sso_domain

In this case we need to identify the stale entry to be removed using the vCenter server logs.

1. Firsboot log will have all the registered service registration entries which are downloaded during the vCenter server installation, from here we can get the valid service ID which was installed in vCenter server.

2. Go to the location /var/log/firstboot in the vCenter server logs and run the command grep -i "Service ID" * | less (service ID for which the duplicate entry is detected)

3. If you get an output as below for the service ID, which means that service ID is currently installed in the vCenter server and it is a valid one which should not be removed.

vmidentity-firstboot.py_2##3_stdout.log:XXXX-0x-XXTXX:XX:XX.XXXZ Running command: ['/usr/lib/vmware-lookupsvc/tools/lstool.py', 'register', '--id', 'Service ID', '--user', 'Administrator@sso_domain', '--password', '*******', '--spec', '/var/lib/lookupsvc/lookupsvc syaas.service.spec', '--url', 'https://vCenter server FQDN.au:443/lookupservice/sdk']

Following steps need to be performed to remove the stale entries identified above:

1. Take a no memory snapshot of the vCenter server if it is standalone or powered off snapshots of vCenter servers if in linked mode

2. Take SSH access to the vCenter server and run the following command to unregister the stale service ID:

/usr/lib/vmware-lookupsvc/tools/lstool.py unregister --url http://localhost:7090/lookupservice/sdk --id Stale_Service_ID --user 'administrator@sso_domain' --password 'sso_password' --no-check-cert

Scenario 2: When the Node ID is different between duplicate service registrations which needs to be removed

Refer the following KB to remove stale entry using a script when the Node ID's are different:

Script to remove stale service registrations from vCenter Server.

Scenario 3: When the Node ID is not present to validate the duplicate service registrations. Also, we do not see any firstboot logs related to the specific service.

Execute the command to list the service registrations on the vCenter.

/usr/lib/vmware-lookupsvc/tools/lstool.py list --url https://localhost/lookupservice/sdk --no-check-cert > /tmp/psc.txt

Check for the number of endpoints for the specific service registration.
Identify which service registration has the least number of endpoints. That specific service ID can be removed as duplicate.

Note: Please ensure that a proper snapshot is taken for the vCenter before making any changes.