NSX manager error while communicating with ESX Agent Manager after VCSA 6.7 upgrade
Article ID: 313020
Updated On:
Products
Issue/Introduction
You experience these symptoms After upgrading to vCenter Server Appliance (VCSA) 6.7:
- The NSX manager error occurred while communicating with ESX Agent Manager
- In the VCSA /var/log/vmware/eam/eam.log file you see entries similar to:
[YYYY-MM-DDTHH:MM:SS] | INFO | vim-monitor | VcConnection.java | 199 | Connecting to vCenter as com.vmware.vim.eam extension
[YYYY-MM-DDTHH:MM:SS] | INFO | vim-monitor | VcConnection.java | 640 | Connecting to https://vcenter01.example.com:8089/sdk/vimService via vCenter proxy http://localhost:80
[YYYY-MM-DDTHH:MM:SS] | ERROR | vim-monitor | VcConnection.java | 653 | Error communicating with the vCenter server.
com.vmware.vim.vmomi.client.common.UnexpectedStatusCodeException: Unexpected status code: 404
at com.vmware.vim.vmomi.client.common.Response$Status.getStatus(Response.java:58)
at com.vmware.vim.vmomi.client.http.impl.HttpExchangeBase.parseResponse(HttpExchangeBase.java:156)
at com.vmware.vim.vmomi.client.http.impl.HttpExchange.run(HttpExchange.java:53)
at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase.executeRunnable(HttpProtocolBindingBase.java:226)
at com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingImpl.send(HttpProtocolBindingImpl.java:106)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.sendCall(MethodInvocationHandlerImpl.java:629)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl$CallExecutor.executeCall(MethodInvocationHandlerImpl.java:610)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.completeCall(MethodInvocationHandlerImpl.java:360)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invokeOperation(MethodInvocationHandlerImpl.java:311)
at com.vmware.vim.vmomi.client.common.impl.MethodInvocationHandlerImpl.invoke(MethodInvocationHandlerImpl.java:184)
at com.sun.proxy.$Proxy32.retrieveContent(Unknown Source)
at com.vmware.eam.vc.VcConnection.retrieveServiceContent(VcConnection.java:648)
at com.vmware.eam.vc.VcConnection.retrieveContentEam(VcConnection.java:550)
at com.vmware.eam.vc.VcConnection.connectEam(VcConnection.java:200)
at com.vmware.eam.vc.VcListener.main(VcListener.java:126)
at com.vmware.eam.vc.VcListener.call(VcListener.java:109)
at com.vmware.eam.vc.VcListener.call(VcListener.java:55)
at com.vmware.eam.async.impl.AuditedJob.call(AuditedJob.java:48)
at com.vmware.eam.async.impl.FutureRunnable.run(FutureRunnable.java:53)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
- You are unable to deploy vSphere Installation Bundles (VIBs) to your ESXi hosts from NSX for vSphere or vCloud Networking and Security.
Environment
VMware NSX
Cause
Endpoints are case-sensitive and references are set to the wrong case in the VMware reverse proxy, vmware-rhttpproxy.
Resolution
Workaround:
1. Ensure you have a valid backup of the vCenter Server and the vCenter Server is powered down if creating a snapshot as a reversion point.
2. Connect to the VCSA command line interface by SSH or console.
3. Change to the directory /etc/vmware-rhttpproxy/endpoints.conf.d
# cd /etc/vmware-rhttpproxy/endpoints.conf.d
4. Find the rhttpproxy.log entries showing how it searches for the VCSA URL. Note the exact case being used for the VCSA FQDN in the output:
# cat /var/log/vmware/rhttpproxy/rhttpproxy.log |grep "8089 does not match"
5. Open the files listed here using a text editor to make the changes below:
vpxd-rhttpproxy-endpoint.conf
vpxd-proxy.conf
vpxd.conf
6. In each of the three files, if you see the below entry, where <vcenter-fqdn> represents your actual VCSA FQDN, change the FQDN case to match what you saw in the rhttpproxy.log:
<vcenter-fqdn>:8089 localtunnel 8089 allow reject
Note: You might not see this entry in vpxd.conf.
Example:
The search of the rhttpproxy.log per the directions above gave us:
[YYYY-MM-DDTHH:MM:SS] info rhttpproxy[48009] [Originator@6876 sub=Proxy Req 00202] URL namespace vcenter01.example.com:8089 does not match any server namespace
Therefore it was searching for a the VCSA FQDN with this case:
vcenter01.example.com
The line in both of the vpxd-proxy.conf and vpxd-rhttpproxy-endpoint.conf files was seen to be:
VCENTER01.example.com:8089 localtunnel 8089 allow reject
That line within each file therefore was changed to:
vcenter01.example.com:8089 localtunnel 8089 allow reject
Feedback
