vCenter Convergence fails with "vCenter.example.com" could not be converged.

Products

VMware vCenter Server

Issue/Introduction

After entering all the required information and credentials for the converge, the task fails with "vCenter.example.com" could not be converged.
The converge process does not even start.
The converge.log does not get created at /var/log/vmware/converge/ folder

Error in /var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log

convertToVcsaEmbedded_Task com.vmware.vapi.std.errors.Unauthorized: Unauthorized `(com.vmware.vapi.std.errors.unauthorized) => {
` messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => {
` id = vapi.security.authorization.invalid,
` defaultMessage = Unable to authorize user,
` args = []
`}],
` data = <null>
`}

In the vSphere Web-client System Configuration, the vCenter server Version shows as "unknown"

Environment

VMware vCenter Server Appliance 6.7.x

Cause

Possible causes:

Domain-name of vmafd is different between the vCenter to be converged.
SSO is not case sensitive but applmgmt does a case sensitive comparison.
/usr/lib/vmware-vmafd/bin/vmafd-cli get-domain-name --server-name localhost on vCenter A will show different than vCenter B.
This name is Case sensitive and should be the same in all nodes to be converged.
It is possible that the CA certificate for the original STS cert (tenantcredential) is no longer present in trusted_roots.

Resolution

Check and compare the SSO domain name following the below commands in all nodes:
1. Log ion to the vCenter with root account
2. /usr/lib/vmware-vmafd/bin/vmafd-cli get-domain-name --server-name localhost in A node shows (VSPHERE.LOCAL)
3. /usr/lib/vmware-vmafd/bin/vmafd-cli get-domain-name --server-name localhost in B node shows (vsphere.local)

Set the domain name same as like in A node by running the following command

/usr/lib/vmware-vmafd/bin/vmafd-cli set-domain-name --server-name localhost --domain-name VSPHERE.LOCAL

Restart applmgmt service with following command

service-control --stop applmgmt && service-control --start applmgmt

Reboot all nodes PSC's and VCSA's
Check the Topology tab in the web client > System Configuration and make sure that the vCenter shows the correct version for all PSC's and VCSA's.

If the version still shows as "unknown" , then follow the below steps

Download and mount the same version of VCSA ISO.
Mount the iso and find the vmware-dns-server rpm:
```
find / -name vmware-dns-server*.rpm
```

Remove the vmware-dns-server rpm

rpm -ev --nodeps vmware-dns-<RPMName-FromStep2>

Note: Run the above command without .rpm extension

Example -

rpm -ev --nodeps vmware-dns-server-1.0.0-12304056.x86_64

Install the vmware-dns-server rpm

rpm -ivh --nodeps /root/velma/rpm/<vmware-dns-name-FromStep2.rpm>

Example -

rpm -ivh --nodeps /root/velma/rpm/vmware-dns-server-1.0.0-12304056.x86_64.rpm

Restart services of the affected node (PSC / VC)
Try converge again.If the issue still persists, then refer to the below KB article and confirm any unused Certificates are present and then unpublish from the PSC/VCSA's to be converged
Removing CA Certificates from the TRUSTED_ROOTS store in the VMware Endpoint Certificate Store(VECS)

Once Cleanup has been performed run command to check the status of the certificates:

for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; done

Trusted Roots Stores should be in a Okay state.
Restart both PSC's and VCSA's services respectively.
Take new Snapshots.
Try the converge again.
If the issue persists collect LDIF's, logs and timestamps and proceed to review them. - Refer How to export VMDir information from vCenter Server Appliance or Platform Service Controller using the JXplorer or ldapsearch

Workaround:

In case the issue persists there are other workarounds which you could try:

Try CLI converge tool.
Try repointing VC to 2nd PSC in same SSO domain(if available) and retry converge.
As a last resort: You could try a Cross-domain repoint and then converge.
- Deploy new PSC in new SSO domain (new hostname).
- Cross-domain repoint VC to new PSC.
- Converge.