Slow vCenter logins when vCenter is attempting to replicate with a decommissioned Enhanced Linked Mode partner
search cancel

Slow vCenter logins when vCenter is attempting to replicate with a decommissioned Enhanced Linked Mode partner

book

Article ID: 413917

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

  • Login to the vCenter using Domain user and Administrator user both of them taking longer then expected. 
  • From the logs, we found that the vCenter was still fetching information from the old vCenter in the Enhanced Linked Mode (ELM) configuration.

var/log/vmware/vmdir/vdcrepadmin.log
YY-MM-DDTHH:MM:SS:ERROR: VmDirAnonymousLDAPBindEx to (ldap:///stalevCenterFQDN:389) failed. (-1)(Can't contact LDAP server)
YY-MM-DDTHH:MM:SS:ERROR: _VmDirGetDSERootAttributeEx failed with error (9127)
YY-MM-DDTHH:MM:SS:ERROR: VmDirGetDomainName failed with error (9127)
YY-MM-DDTHH:MM:SS:WARNING: VmDirGetReplicationPartnerStatus, partner (/stalevCenterFQDN) not available (9127)
YY-MM-DDTHH:MM:SS:ERROR: VmDirAnonymousLDAPBindEx to (ldap:///stalevCenterFQDN:389) failed. (-1)(Can't contact LDAP server)

var/log/vmware/rhttpproxy.log/rhttpproxy.log
YY-MM-DDTHH:MM:SS: info rhttpproxy[3553982] [Originator@6876 sub=RhttpProxy] [Envoy GET Handler] URI: /clusters?uri=/external-vecs/http1///stalevCenterFQDN/443/apigw
YY-MM-DDTHH:MM:SS: info rhttpproxy[3553982] [Originator@6876 sub=RhttpProxy] [Envoy GET Handler] URI: /clusters?uri=/external-vecs/http1///stalevCenterFQDN/443/sdk
YY-MM-DDTHH:MM:SS: info rhttpproxy[3553982] [Originator@6876 sub=RhttpProxy] [Envoy GET Handler] URI: /clusters?uri=/external-vecs/http1///stalevCenterFQDN/443/sdk
YY-MM-DDTHH:MM:SS: info rhttpproxy[3553930] [Originator@6876 sub=RhttpProxy.HTTPService.HttpConnection] HTTP Connection has timed out while waiting for further requests; <io_obj p:0x00007f4e240fd580, h:-1, <TCP '127.0.0.1 : 549'>, <TCP '127.0.0.1 : 52356'>>, N7Vmacore16TimeoutExceptionE(Operation timed out: Stream: <io_obj p:0x00007f4e240fd580, h:-1, <TCP '127.0.0.1 : 549'>, <TCP '127.0.0.1 : 52356'>>, duration: 00:00:46.420561 (hh:mm:ss.us))

/var/log/vmware/dnsmasq.log 
YY-MM-DDTHH:MM:SS dnsmasq[2096]: query[A] //stalevCenterFQDN from 127.0.0.1
YY-MM-DDTHH:MM:SS dnsmasq[2096]: cached //stalevCenterFQDN is NXDOMAIN
YY-MM-DDTHH:MM:SS dnsmasq[2096]: query[A] //stalevCenterFQDN from 127.0.0.1
YY-MM-DDTHH:MM:SS dnsmasq[2096]: cached //stalevCenterFQDN is NXDOMAIN
YY-MM-DDTHH:MM:SS dnsmasq[2096]: query[A] //stalevCenterFQDN from 127.0.0.1
YY-MM-DDTHH:MM:SS dnsmasq[2096]: cached //stalevCenterFQDN is NXDOMAIN
YY-MM-DDTHH:MM:SS dnsmasq[2096]: query[A] //stalevCenterFQDN from 127.0.0.1
YY-MM-DDTHH:MM:SS dnsmasq[2096]: cached //stalevCenterFQDN is NXDOMAIN

Environment

VMware vCenter Server

Cause

Slow vCenter logins may occur when the vCenter is still attempting to replicate with a decommissioned Enhanced Linked Mode partner, which can delay authentication.

Resolution

IMPORTANT! Snapshot the vCenter before doing any data removal procedures. See "Snapshot Best practices for vCenter Server Virtual Machines" and "VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practice" for more information.

Displays all of the linked vCenter servers within a vSphere domain by following command. 

  • /usr/lib/vmware-vmdir/bin/vdcrepadmin -f showservers -h localhost -u administrator

         (Note: SSO domain password administrator@your_domain_name )

example output 

cn=vc1.example.local,cn=Servers,cn=home,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=vc2.example.local,cn=Servers,cn=home,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=vc3.example.local,cn=Servers,cn=home,cn=Sites,cn=Configuration,dc=vsphere,dc=local
cn=vc4.example.local,cn=Servers,cn=home,cn=Sites,cn=Configuration,dc=vsphere,dc=local

  • From the listed vCenter servers verify the decommissioned partner vCenter.
  • Use either of the following options to resolve the issue:

Option 1: 

Process to Decommission a Node from the vSphere Domain and Inventory

Before following below command ensure that Partner_node_System vCenter server in Power off state.

cmsso-util unregister --node-pnid Partner_node_System_Name --username administrator@your_domain_name --passwd 'vCenter_Single_Sign_On_password'

Partner_node_System_Name  = Decommissioned partner vCenter FQDN

Option 2: 

To resolve the issue, re-point the vCenter Server to itself by running the command described in the Splitting Enhanced Linked Mode (ELM) KB article

cmsso-util domain-repoint -m execute --src-emb-admin Administrator --dest-domain-name vsphere.local

  • Removing decommissioned partner node OR  Repointing the vCenter to itself remove the linked references  resolving the issue and enabling faster logins.

Additional Information

Process to Decommission a Node from the vSphere Domain and Inventory

Splitting Enhanced Linked Mode (ELM) 

https://knowledge.broadcom.com/external/article/385788/vcenter-is-slow-to-login-to-when-one-or.html