PNID change fails and vCenter services fail to start due to duplicate Domain Controller entries

Products

• VMware vCenter Server

Issue/Introduction

 

• Attempts to change the Primary Network Identifier (PNID) of the vCenter Server fail.

• The vSphere Profile-Driven Storage Service (SPS) and other STS-dependent services fail to start.

• Reviewing the Identity Management or STS logs reveals the following error indicating duplicate entries:

  java.lang.IllegalStateException: Internal error : duplicate entries were found
at com.vmware.identity.idm.server.provider.vmwdirectory.VMwareDirectoryProvider.getAttributes

   

 

Environment

VMware vCenter Server 8.x

Cause

This issue occurs when stale or duplicate Domain Controller records (such as an old IP address acting as a previous PNID) are present in the ou=Domain Controllers container within the VMware Directory Service (vmdir) database. When the Identity Management (IDM) service queries for attributes during the PNID change, the multiple results trigger an IllegalStateException, halting the workflow.

Resolution

IMPORTANT: Ensure you have offline snapshots (powered off) or file-based backups of all vCenter Servers in the SSO domain before proceeding with manual database edits.

You can resolve this issue manually using the command line or by running a provided shell script.

Option 1: Manual Command Line Method

Step 1: Identify the duplicate Domain Controller entry

1. SSH into the vCenter Server Appliance (VCSA) and log in as root.

2. Run the following ldapsearch command to list all Domain Controller objects. (Note: Replace vsphere.local with your SSO domain and <SSO_Password> with your [email protected] password).

   Bash

    

   /opt/likewise/bin/ldapsearch -LLL -h localhost -p 389 -b "ou=Domain Controllers,dc=vsphere,dc=local" -s sub dn -D "cn=administrator,cn=users,dc=vsphere,dc=local" -o ldif-wrap=no -w "<SSO_Password>"
   

3. Review the output. It will look similar to this:

   Plaintext

    

   dn: ou=Domain Controllers,dc=vsphere,dc=local
   dn: cn=<VCIP>,ou=Domain Controllers,dc=vsphere,dc=local
   dn: cn=<VCFQDN>,ou=Domain Controllers,dc=vsphere,dc=local
   dn: cn=<VCFQDN>,ou=Domain Controllers,dc=vsphere,dc=local
   

4. Identify the entry representing the "old" PNID (e.g., cn=<VCIP>).

Step 2: Delete the duplicate entry

1. Run the ldapdelete command to recursively remove the stale entry identified in Step 1.

   Bash

    

   /opt/likewise/bin/ldapdelete -r -h localhost -p 389 "cn=<VCIP>,ou=Domain Controllers,dc=vsphere,dc=local" -D "cn=administrator,cn=users,dc=vsphere,dc=local" -w "<SSO_Password>"
   

2. Re-run the search command from Step 1 to verify the stale entry has been successfully removed.

3. Restart vCenter services or resume the PNID change workflow.

---

Option 2: Scripted Method

If you prefer to automate the search and cleanup process, download the attached remove_stale_pnid.sh script and execute it.

STEPS TO USE THE SCRIPT: 

         1)  Use copy/paste to copy the attached "remove_stale_pnid.sh" script to the vCenter Server using VI editor or use a utility like WinSCP to copy the attached "remove_stale_pnid.sh"                      script to a directory of your choice on the vCenter Server.

         2)  Once the script has been copied to the vCenter Server run a chmod command to make it executable (For this example, the script is located in the /tmp directory)
       chmod +x /tmp/remove_stale_pnid.sh



        3)  Execute the script with the following command (For this example, the script is being executed from the /tmp directory):

             ./tmp/remove_stale_pnid.sh

             You will be prompted to enter the SSO Admin password.



 

        4) When the script runs you will be presented with a list of domain controllers

5) Follow the on-screen prompts to view the domain controllers, input the old PNID number(0 in our case), and confirm the deletion.




6) Once the script finishes, it shows the below screen, which means the duplicate entry of domain controller is removed.



7) Restart vCenter services or resume the PNID change workflow.