Upgrade to vCenter Server 8.0.x fails with "ESX Agent Manager (EAM) upgrade failed to obtain EAM URLs"
search cancel

Upgrade to vCenter Server 8.0.x fails with "ESX Agent Manager (EAM) upgrade failed to obtain EAM URLs"

book

Article ID: 326214

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms:

  • Upgrade to vCenter Server 8.0.x fails with the following error during pre-check: 
    Source vSphere ESX Agent Manager (EAM) upgrade failed to obtain EAM URLs to check against trusted
certificates by the System!

Verify that the ESX Agent Manager extension is running properly on the source vCenter Server instance
and https://VC_IP/eam/mob presents correct data. If log in to the MOB is not successful, try resolving
the issue with https://kb.vmware.com/s/article/94934.
  • for example:
  • when looking in /var/log/vmware/eam/eam.log in the source vCenter Server, the following errors can be found: 
    eam.log
.. [YYYY-MM-DDTHH:MM:SS] |  INFO | vim-monitor | VcListener.java | 125 | Full stack trace: com.vmware.eam.EamRemoteSystemException: Client error communicating with the > vCenter server.
at com.vmware.eam.vim.server.impl.VimRoot.rootOperation(VimRoot.java:103)
at com.vmware.eam.vim.server.impl.VimRoot.currentTime(VimRoot.java:78)
..
Caused by: com.vmware.vim.vmomi.client.common.UnexpectedStatusCodeException: Unexpected status code: 404
at com.vmware.vim.vmomi.client.common.Response$Status.getStatus(Response.java:56)
at com.vmware.vim.vmomi.client.http.impl.HttpExchangeBase.parseResponse(HttpExchangeBase.java:271)
at com.vmware.vim.vmomi.client.http.impl.HttpExchange.invokeWithinScope(HttpExchange.java:54)
at com.vmware.vim.vmomi.client.http.impl.TracingScopedRunnable.run(TracingScopedRunnable.java:24)
at com.vmware.vim.vmomi.client.http.impl.HttpExchangeBase.run(HttpExchangeBase.java:57)
at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase.executeRunnable(HttpProtocolBindingBase.java:227)
at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:114)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.sendCall(MethodInvocationHandlerImpl.java:693)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.executeCall(MethodInvocationHandlerImpl.java:674)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.completeCall(MethodInvocationHandlerImpl.java:371)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invokeOperation(MethodInvocationHandlerImpl.java:322)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invoke(MethodInvocationHandlerImpl.java:195)
at com.sun.proxy.$Proxy50.currentTime(Unknown Source)
at com.vmware.eam.vim.server.impl.VimRoot.rootOperation(VimRoot.java:101)
... 9 more
..

 

  • When querying the PNID on the source vCenter, you find it to be in small letters: 
    # /usr/lib/vmware-vmafd/bin/vmafd-cli get-pnid --server-name localhost
vcsa.domain.local
  • However, when looking in /etc/vmware-rhttpproxy/endpoints.conf.d/ProxyConfiguration.json in the source vCenter, its hostname portion is spelled in upper-case: 
    VCSA.domain.local



Environment

VMware vCenter Server 7.0.x

VMware vCenter Server 8.0.x

Cause

This issue occurs, when there is a difference in the case-sensitivity between the vCenters FQDN, its primary network identifier (PNID), the domain controller name (DCName) of its embedded platform service controller in the vsphere.local domain, and the spelling of the PNID in /etc/vmware-rhttpproxy/endpoints.conf.d/ProxyConfiguration.json in vCenter Server 7.0.

In vCenter Server 8.x, /etc/vmware-rhttpproxy/endpoints.conf.d/ProxyConfiguration.json no longer contains the vCenter FQDN, but replaces it with localhost.

Resolution

Caution: Before applying the steps below, please take a backup or an offline-snapshot (in powered-off state) of the vCenter Server Appliance . If the vCenter is part of a Linked Mode replication setup, please backup/snapshot all replicating nodes as well.

To resolve this issue, change the PNID of the vCenter Server to correspond with the spelling in /etc/vmware-rhttpproxy/endpoints.conf.d/ProxyConfiguration.json. However, since you cannot do a direct change to the same PNID in a different case, this is a 2-step process:
  1. Create a temporary record in your DNS server with the current vCenter IP and a temporary FQDN (e.g. VCSA1.domain.local)
  2. Change the PNID to this temporary FQDN following the complete process outlined in Reconfigure the Primary Network Identifier
  3. Do a second PNID change back to the old FQDN, but using the spelling that was found in /etc/vmware-rhttpproxy/endpoints.conf.d/ProxyConfiguration.json. For example, if the vCenter FQDN is listed as VCSA.mj.local in ProxyConfiguration.json, use this as the new PNID
  4. remove the temporary DNS record
  5. Upgrade the vCenter Server

Additional Information

If you are still getting the pre-upgrade check failure referencing EAM URLs, or the error messages in /var/log/vmware/eam/eam.log mentioning "Client error communicating with the > vCenter server", then you may need to run lsdoctor.py --trustfix, in order to update the service trusts (which includes EAM's aforementioned URL's).

For more information about running lsdoctor, please refer to Using the 'lsdoctor' Tool.

 

 

Powered by Wolken Software