Unable to login to a Linked Mode vCenter Server using SSO domain credentials after upgrading to vCenter 7.0

Products

• VMware vCenter Server 7.0

Issue/Introduction

During an upgrade of a vCenter Server in ELM to any release of vCenter 7.0, logins using the system domain may fail under certain conditions. The system domain is typically vsphere.local.  This problem typically affects servers in Enhanced Link Mode groups, but may also affect an individual vCenter deployed with Cloud Gateway.

This KB article is strictly for fixing a vCenter Server in ELM. For information on repairing a vCenter Cloud Gateway, see KB 367227.

Environment

vCenter Server 7.0.x

Cause

During the upgrade, the system domain's identity source is redirected to the system domain of the most recently installed server.  If that server is immediately taken offline, then NO servers can be logged into using the system domain identity source.  This problem occurs when the vCenter is taken offline during the upgrade process before completing it.  Note that upgrades irrevocably upgrade the schema, so once an upgrade is started, it must be finished expediently.

Resolution

To avoid this situation, upgrade all vCenter Servers in an ELM group before taking them offline.  If this cannot be avoided, use the workaround below to update the vmwSTSConnectionStrings value on the affected vCenter Server.

 

Preparation

1. Connect to the Cloud Gateway via SSH.

2. Construct the LDAP distinguished name for the Administrator account.  For example, if the system domain is vsphere.local, the distinguished name is:

cn=Administrator, cn=Users, dc=vsphere, dc=local

3. Construct the LDAP distinguished name for the system domain configuration entry.  For example, if the system domain is vsphere.local, the distinguished name is:

cn=vsphere.local,cn=IdentityProviders,cn=vsphere.local,cn=Tenants,cn=IdentityManager,cn=Services,dc=vsphere,dc=local

4. Display the system domain definition on the local machine by entering the following command, then providing the password when prompted:

ldapsearch -h localhost -D "<Administrator LDAP Distinguished Name>" -W -b "<System domain configuration entry LDAP distinguished name>" -s base

For example, for the default domain of vsphere.local, the command is:

ldapsearch -h localhost -D "cn=Administrator, cn=Users, dc=vsphere, dc=local" -W -b "cn=vsphere.local,cn=IdentityProviders,cn=vsphere.local,cn=Tenants,cn=IdentityManager,cn=Services,dc=vsphere,dc=local" -s base

 

Repair

1. Scan the output, verifying that the vmwSTSConnectionStrings attribute is equal or to the FQDN of a vCenter taken offline. If so, continue.

dn: cn=vsphere.local,cn=IdentityProviders,cn=vsphere.local,cn=Tenants,cn=IdentityManager,cn=Services,dc=vsphere,dc=local
objectClass: top
objectClass: vmwSTSIdentityStore
cn: vsphere.local
nTSecurityDescriptor:: AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAA
vmwSTSIdentityStoreFlags: 0
vmwSTSGroupBaseDN: DC=vsphere,DC=local
vmwSTSUserBaseDN: DC=vsphere,DC=local
vmwSTSConnectionStrings: ldap://example-vcenter.example.com:389
vmwSTSServiceUseMachineAccount: false
vmwSTSUserName: [email protected]
vmwSTSTimeout: 0
vmwSTSAuthenticationType: SRP
vmwSTSProviderType: IDENTITY_STORE_TYPE_VMWARE_DIRECTORY
vmwSTSDomainName: vsphere.local
vmwSTSDomainType: SYSTEM_DOMAIN

2. Create an LDIF file to redirect vmwSTSConnectionStrings to localhost by entering the following commands.  Replace the dn: value with the LDAP distinguished name for the system domain configuration entry created in Preparation Step 3.

echo 'dn: cn=vsphere.local,cn=IdentityProviders,cn=vsphere.local,cn=Tenants,cn=IdentityManager,cn=Services,dc=vsphere,dc=local' > fix.ldif
echo 'changetype: modify' >> fix.ldif
echo 'replace: vmwSTSConnectionStrings' >> fix.ldif
printf 'vmwSTSConnectionStrings: ldap://localhost:389' >> fix.ldif

3. Repair the system domain by entering the following command, replacing the -D parameter with the Administrator account LDAP distinguished name created in Preparation Step 2.  Provide the password when prompted.

ldapmodify -h localhost -D "cn=Administrator, cn=Users, dc=vsphere, dc=local" -W -f fix.ldif

4. Verify the repair by re-running the command from Preparation Step 4 and making sure vmwSTSConnectionStrings value is equal to "ldap://localhost:389".

5. Restart vCenter, then login to the GUI.

Additional Information

https://www.rfc-editor.org/rfc/rfc4514