vCenter not visible in Enhanced Linked Mode

Products

• VMware vCenter Server

Issue/Introduction

• After upgrade/patch of vCenter in ELM environment failed to display the partner vCenter on the UI
• Checking partner status following Determining replication agreements and status with the Platform Services Controller (PSC)  returns 0 changes behind.
• On vCenter that is not seeing missing node you see following in logs :
• In var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log 

 [INFO ] vim-authentication-pool-432  70001045 100086 200037 c.v.vise.vim.commons.extensions.impl.Linked
VcGroupRegistryImpl    Added a linked VC group to registry. Endpoints: https://###.###.##:443/sdk(33333#####,vsphere.local(44444#####)),https://###.###.##:443/sdk(55555######,vsphere.local(66666#########)),

• The task references all visible nodes being added to the group. FQDNs and node IDs are unique to each environment
• In var/log/vmware/vsphere-ui/logs/catalina.log :

[http-nio-127.0.0.1-5090-exec-8] com.vmware.vum.client.exceptions.GlobalExceptionHandler.logException ServiceUnavailable (com.vmware.vapi.std.errors.service_unavailable) =>
 {
    messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => {
    id = apigw.error.vcenterIdNotFound,
    defaultMessage = vCenter not found: 11111#####
    args = [],
    params = <null>,
    localized = <null>
}],
    data = <null>,
    errorType = SERVICE_UNAVAILABLE
}

• Checking logs on vCenter that see all nodes in var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log
• This task will list all nodes in environment:

[INFO ] vim-authentication-pool-432  70001045 100086 200037 c.v.vise.vim.commons.extensions.impl.Linked
VcGroupRegistryImpl    Added a linked VC group to registry. Endpoints: https://###.###.##:443/sdk(11111#####,vsphere.local(22222#####)),https://###.###.##:443/sdk(33333######,vsphere.local(44444#########)),https://###.###.##:443/sdk(55555#####,,vsphere.local(66666#####))

This task will reference missing vCenter and it will contain same ID mentioned in catalina.log on vCenter that is not seeing all nodes.

Environment

vCenter Server 8.0.x

Cause

The service registration  "vcenterserver"  is missing in VMDir 

 

Resolution

1. In many post-patching scenarios where Enhanced Linked Mode (ELM) synchronization appears broken, a simple reboot of the patched vCenter is sufficient to re-initialize the core vCenter management services and restore active synchronization across the ELM partner nodes.
   
   
   1. Connect to the patched vCenter using SSH and type shell and press Enter
   2. Reboot the patched vCenter Server.
   3. Allow services to fully initialize (approx. 5–10 minutes).
   4. Check the ELM connectivity status in the vSphere Client.
2. If the vCenter remains missing from the UI after a full reboot, proceed with the troubleshooting steps below.

1. 1. Connect to the affected vCenter using SSH and type shell and press Enter
   2. To create a text file with a list of the services registered within the vCenters , run this command: 
      

      /usr/lib/vmware-lookupsvc/tools/lstool.py list --url http://localhost:7090/lookupservice/sdk --type vcenterserver > /tmp/psc_services.txt --no-check-cert

   3. In the generated file search for vcenterserver services :
      

      grep "vcenterserver" /tmp/psc_services.txt

   4. Ensure every vCenter Server in the Linked Mode environment has a valid vcenterserver service registration. If any node is missing its registration, execute fixpsc rebuild against a healthy partner node.
      Fix PSC/vmdir inconsistencies using fixpsc python script