Recreate vCenter Server Machine Account in Platform Services Controller after Failed Convergence

Products

• VMware vCenter Server

Issue/Introduction

This article will help to recreate the vCenter Server Machine account on a failed Convergence environment

Symptoms:

• After a failed attempt to converge a vCenter Server and external PSC, vCenter services won't start and you see the following in the VMDIR log /var/log/vmware/vmdird/vmdird-syslog.log on the PSC:

19-05-15T20:27:41.456363+00:00 err vmdird  t@###############: VmDirSRPGetIdentityData ([email protected]) failed, (9611)
19-05-15T20:27:41.456744+00:00 err vmdird  t@###############: VmDirSRPGetIdentityData ([email protected]) failed, (9611)
19-05-15T20:27:41.457011+00:00 err vmdird  t@###############: SASLSessionStart: sasl error (-20)(SASL(-13): user not found: no secret in database)
19-05-15T20:27:41.457268+00:00 err vmdird  t@###############: VmDirSendLdapResult: Request (Bind), Error (49), Message ((49)(SASL start failed.)), (0) socket (192.168.0.71)
19-05-15T20:27:41.457582+00:00 err vmdird  t@###############: Bind Request Failed (192.168.0.71) error 49: Protocol version: 3, Bind DN: "", Method: SASL
19-05-15T20:27:42.466907+00:00 err vmdird  t@###############: VmDirSRPGetIdentityData ([email protected]) failed, (9611)
19-05-15T20:27:42.467344+00:00 err vmdird  t@###############: VmDirSRPGetIdentityData ([email protected]) failed, (9611)
19-05-15T20:27:42.467596+00:00 err vmdird  t@###############: SASLSessionStart: sasl error (-20)(SASL(-13): user not found: no secret in database)
19-05-15T20:27:42.467823+00:00 err vmdird  t@###############: VmDirSendLdapResult: Request (Bind), Error (49), Message ((49)(SASL start failed.)), (0) socket (192.168.0.71)

• There are no backups or powered-off snapshots of all PSC's and VC's in the environment.  No formal recovery option is available.

Environment

VMware vCenter Server 7.0.x
VMware vCenter Server 6.x

Cause

As part of the converge process, the VC computer account is removed is removed from PSC, and a domain controller account is created (for the new embedded PSC).  If the converge fails after this point, the rollback will not recreate the computer account.

Resolution

NOTE:  Please take offline (powered off) snapshots of all PSC's and VC's before attempting.  This is standard best practice before making any manual changes to the PSC VMDIR database.      

1. Copy the script attached to this article on the vCenter Server where converge operation failed.
2. Make the script executable:
   • chmod +x recreate_machine.sh
3. Execute the script.  You will be prompted for the Administrator@<sso.domain> password. 

Example output :

root@vc1 [ ~ ]# ./recreate_machine.sh
==================================
Computer object creation for vc1.example.com started on Fri May 17 18:18:53 UTC 2019


Detected site GUID: ########
Detected machine GUID: #######
Detected DN: cn=vc1.example.com,ou=Computers,dc=vsphere,dc=local
Detected CN: vc1.example.com
Detected SSO domain: vsphere.local
Detected principal name: [email protected]
Detected PSC name: psc1.example.com

Enter SSO admin password:
Creating computer account object...
adding new entry "cn=vc1.example.com,ou=Computers,dc=vsphere,dc=local"

Generated password: ############
Applying password to registry.
Applying password to vmdir.
modifying entry "cn=vc1.example.com,ou=Computers,dc=vsphere,dc=local"

Adding cn=vc1.example.com,ou=Computers,dc=vsphere,dc=local to DCClients group...
modifying entry "cn=DCClients,cn=Builtin,dc=vsphere,dc=local"

Done on Fri May 17 18:18:57 UTC 2019.  You may need to restart services...
 

Note:  You may receive an error when you try to run the script:
bash:  ./recreate_machine.sh: /bin/bash^M: bad interpreter: No such file or directory
This error is caused by DOS carriage returns added to the script when copying from a Windows based text editor.  To resolve this problem, run the following command and rerun the script:

sed -i -e 's/\r$//' recreate_machine.sh
 

Additional Information

If you encounter any issues during the script execution, contact Broadcom Support.

Impact/Risks:
The article assumes you have taken powered off snapshots of all PSCs and vCenter Servers in ELM prior to attempting the fix (per the instructions set forth in the resolution section of this article). Should something go wrong, you will have to restore to the snapshots taken before the attempted fix.