NSX manager error while communicating with esxagaentmanager after VCSA 6.7 upgrade
search cancel

NSX manager error while communicating with esxagaentmanager after VCSA 6.7 upgrade

book

Article ID: 313020

calendar_today

Updated On:

Products

VMware vCenter Server VMware NSX

Issue/Introduction

You experience these symptoms After upgrading to vCenter Server Appliance (VCSA) 6.7:

  • The  NSX manager error occurred while communicating with esxagaentmanager 
  • In the VCSA /var/log/vmware/eam/eam.log file you see entries similar to:

2020-02-29T14:39:33.285+02:00 |  INFO | vim-monitor | VcConnection.java | 199 | Connecting to vCenter as com.vmware.vim.eam extension
2020-02-29T14:39:33.295+02:00 |  INFO | vim-monitor | VcConnection.java | 640 | Connecting to https://vcenter01.example.com:8089/sdk/vimService via vCenter proxy http://localhost:80
2020-02-29T14:39:33.297+02:00 | ERROR | vim-monitor | VcConnection.java | 653 | Error communicating with the vCenter server.
com.vmware.vim.vmomi.client.common.UnexpectedStatusCodeException: Unexpected status code: 404
        at com.vmware.vim.vmomi.client.common.Response$Status.getStatus(Response.java:58)
        at com.vmware.vim.vmomi.client.http.impl.HttpExchangeBase.parseResponse(HttpExchangeBase.java:156)
        at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:53)
        at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase.executeRunnable(HttpProtocolBindingBase.java:226)
        at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:106)
        at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.sendCall(MethodInvocationHandlerImpl.java:629)
        at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.executeCall(MethodInvocationHandlerImpl.java:610)
        at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.completeCall(MethodInvocationHandlerImpl.java:360)
        at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invokeOperation(MethodInvocationHandlerImpl.java:311)
        at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invoke(MethodInvocationHandlerImpl.java:184)
        at com.sun.proxy.$Proxy32.retrieveContent(Unknown Source)
        at com.vmware.eam.vc.VcConnection.retrieveServiceContent(VcConnection.java:648)
        at com.vmware.eam.vc.VcConnection.retrieveContentEam(VcConnection.java:550)
        at com.vmware.eam.vc.VcConnection.connectEam(VcConnection.java:200)
        at com.vmware.eam.vc.VcListener.main(VcListener.java:126)
        at com.vmware.eam.vc.VcListener.call(VcListener.java:109)
        at com.vmware.eam.vc.VcListener.call(VcListener.java:55)
        at com.vmware.eam.async.impl.AuditedJob.call(AuditedJob.java:48)
        at com.vmware.eam.async.impl.FutureRunnable.run(FutureRunnable.java:53)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
       at java.lang.Thread.run(Thread.java:748)

 

  • You are unable to deploy vSphere Installation Bundles (VIBs) to your ESXi hosts from NSX for vSphere or vCloud Networking and Security.

Cause

Endpoints are case-sensitive and references are set to the wrong case in the VMware reverse proxy, vmware-rhttpproxy.

Resolution

Workaround:

1. Ensure you have a valid backup of the vCenter Server and the vCenter Server is powered down if creating a snapshot as a reversion point.

2. Connect to the VCSA command line interface by SSH or console.

3. Change to the directory /etc/vmware-rhttpproxy/endpoints.conf.d
# cd /etc/vmware-rhttpproxy/endpoints.conf.d

4. Find the rhttpproxy.log entries showing how it searches for the VCSA URL. Note the exact case being used for the VCSA FQDN in the output:
# cat /var/log/vmware/rhttpproxy/rhttpproxy.log |grep "8089 does not match"

5. Open the files listed here using a text editor to make the changes below: 

vpxd-rhttpproxy-endpoint.conf

vpxd-proxy.conf

vpxd.conf

In each of the three files, if you see the below entry, where <vcenter-fqdn> represents your actual VCSA FQDN, change the FQDN case to match what you saw in the rhttpproxy.log:

<vcenter-fqdn>:8089 localtunnel 8089 allow reject

Note: You might not see this entry in vpxd.conf.

Example:
The search of the rhttpproxy.log per the directions above gave us:

2020-01-30T12:25:35.495+06:00 info rhttpproxy[48009] [Originator@6876 sub=Proxy Req 00202] URL namespace vcenter01.example.com:8089 does not match any server namespace

Therefore it was searching for a the VCSA FQDN with this case:
vcenter01.example.com

The line in both of the vpxd-proxy.conf and vpxd-rhttpproxy-endpoint.conf files was seen to be:
VCENTER01.example.com:8089 localtunnel 8089 allow reject

That line within each file therefore was changed to:
vcenter01.example.com:8089 localtunnel 8089 allow reject