Unable to see other ELM vCenter Servers in inventory
search cancel

Unable to see other ELM vCenter Servers in inventory

book

Article ID: 305288

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

  • There are two or more vCenter Servers in a Enhanced Linked Mode (ELM)
  • When logged into the web client for vCenterA, the user sees vCenterA and vCenterB in the inventory, but when logged into vCenterB, the user only sees vCenterB in the inventory (vCenterA is not shown).
  • In the vCenter - /var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log file, log entries display:

    [<YYYY-MM-DD>T<TIME>] [INFO ] aggregator-query-service-152 70000066 100004 200001 com.vmware.vise.search.transport.impl.AggregatorQueryServiceImpl QueryService LinkedQueryService(https://<vCenterFQDN>:443/invsvc) failed to respond: com.vmware.vise.search.transport.HostConnectException: Unable to connect to VMware Inventory Service -(https://<vCenterFQDN>:443/invsvc). 
com.vmware.vise.search.transport.HostConnectException: Unable to connect to VMware Inventory Service - (https://vcenterfqdn:443/invsvc)
    at com.vmware.vise.util.concurrent.ExecutorUtil$2.run(ExecutorUtil.java:195)
    at com.vmware.vise.util.concurrent.ExecutorUtil$ThreadContextPropagatingRunnable.run(ExecutorUtil.java:928)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
    at java.lang.Thread.run(Thread.java:745)
Caused by: com.vmware.vim.vmomi.client.exception.SslException: com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain not verified
    at com.vmware.vim.vmomi.client.common.impl.ResponseImpl.setError(ResponseImpl.java:251)

Cause

One of the vCenter Servers in the ELM environment cannot verify the certificate chain of another vCenter Server. Due to this, the connection fails and as a result the other vCenter Servers are not shown in the inventory. 

Resolution

Review the certificates in the VMware Endpoint Certificate Store (VECS) to verify that the signing certificates of the other ELM vCenter Server nodes are present. If any are missing, import the missing information.

  1. SSH into the vCenter which cannot see one or more of the ELM vCenter Servers nodes.
  2. Run the command below to get the output of the TRUSTED_ROOTS certificate store in the VECS:

/usr/lib/vmware-vmafd/bin/vecs-cli entry list --store TRUSTED_ROOTS --text | grep -E "Subject:|Issuer:|Not Before:|Not After|Alias"

Example Output

Alias : 9e#######c89d0c5c7d0aa####52c19e7913
        Issuer: CN=<FDQN of vCenterA>, DC=vsphere, DC=local, C=US, ST=California, O=<FDQN of vCenterA>, OU=VMware Engineering
            Not Before: <MM DD hh:mm:ss YYYY> GMT
            Not After : <MM DD hh:mm:ss YYYY> GMT
        Subject: CN=<FDQN of VCSA2>, DC=vsphere, DC=local, C=US, ST=California, O=<FDQN of vCenterA>, OU=VMware Engineering

Alias : 5######8ffdd3d508652855b9######0e0c51d19
        Issuer: CN=<FDQN of vCenterB>, DC=vsphere, DC=local, C=US, ST=California, O=<FDQN of vCenterB>, OU=VMware Engineering
            Not Before: <MM DD hh:mm:ss YYYY> GMT
            Not After : <MM DD hh:mm:ss YYYY> GMT
        Subject: CN=<FDQN of VCSA1>, DC=vsphere, DC=local, C=US, ST=California, O=<FDQN of vCenterB>, OU=VMware Engineering

Note: If the Issuer and Subject are the same, this indicates that it is a signing or root certificate.
  1. Review the CN value of the Issuer string for each signing certificate and check whether one signing certificate is present for each of the other nodes.
  2. Next, connect to each of the nodes for which the certificate is missing and export the signing certificate from the TRUSTED_ROOTS store. 
  1. From any node, import the missing signing certs into the VMware Directory Service (VMDIR) using the command below:

/usr/lib/vmware-vmafd/bin/dir-cli trustedcert publish --cert <PATH_TO_CERTIFICATE> --login <VSPHERE_ADMINISTRATOR>

Example

/usr/lib/vmware-vmafd/bin/dir-cli trustedcert publish --cert /tmp/certs/vCenterA_root.crt --login [email protected]

  1. Force a push of all certificates in VMDIR to the VECS on each node in the ELM environment using the command:

    /usr/lib/vmware-vmafd/bin/vecs-cli force-refresh

  2. Log out and back into the web client to confirm that all nodes of the ELM are now showing in the vCenter Server inventory again. 

Additional Information

Powered by Wolken Software