Error: "Bad Request: Error occurred while fetching data., error code 60506" when loading the IPsec VPN page

Products

• VMware Cloud Director
• VMware NSX

Issue/Introduction

• In the Cloud Director Tenant UI screen "Datacenters -> <Datacenter name> -> Networking -> Edges -> Services -> IPsec VPN" an error occurs stating:

"Error <request id> Bad Request: Error occurred while fetching data., error code 60506"
  
  
• The 'NEW" button is missing from the "IPsec VPN" UI screen.
• API GET to /cloudapi/2.0.0/edgeGateways/urn:vcloud:gateway:<gateway-id>/ipsec/tunnels fails with HTTP 400 Bad Request.
• IPsec connections disappear.
• Within the Cloud Director log file /opt/vmware/vcloud-director/logs/vcloud-container-debug.log the following is observed:
   
  com.vmware.vcloud.common.network.VsmException: Bad Request: Error occurred while fetching data., error code 60506
com.vmware.vcloud.networking.model.nsxtresource.NsxTExceptionUtil.convertToVsmException(NsxTExceptionUtil.java:139)
com.vmware.vcloud.networking.model.nsxtresource.NsxTExceptionUtil.runAndTranslateToVsmException(NsxTExceptionUtil.java:97)
com.vmware.vcloud.networking.model.nsxtresource.NsxTSearch.queryLocalManager(NsxTSearch.java:59) 
com.vmware.vcloud.networking.model.nsxtresource.NsxTSearch.query(NsxTSearch.java:54)
com.vmware.vcloud.networking.model.nsxtresource.NsxTSearch.query(NsxTSearch.java:43) 
com.vmware.vcloud.networking.model.nsxtresource.ipsecvpn.NsxTIpSecVpnService.getIpSecVpnSessionsViaSearch(NsxTIpSecVpnService.java:376) com.vmware.vcloud.networking.model.nsxtresource.ipsecvpn.NsxTIpSecVpnService.lambda$getConfiguredTunnels$0(NsxTIpSecVpnService.java:214) com.vmware.vcloud.networking.services.clients.cursor.NetworkingCursorPaginator.paginateThroughAll(NetworkingCursorPaginator.java:102) 
com.vmware.vcloud.networking.model.nsxtresource.ipsecvpn.NsxTIpSecVpnService.getConfiguredTunnels(NsxTIpSecVpnService.java:214) com.vmware.vcloud.networking.model.commongateway.AbstractNsxTGatewayIpSecVpn.getIpSecVpnTunnelsPage(AbstractNsxTGatewayIpSecVpn.java:607) com.vmware.vcloud.networking.model.commongateway.AbstractNsxTGatewayIpSecVpn.getIpSecVpnTunnels(AbstractNsxTGatewayIpSecVpn.java:596) com.vmware.vcloud.networking.services.handlerproxy.EdgeGatewayIpSecVpnHandlerProxyImpl.getPaginatedIpSecVpnTunnels(EdgeGatewayIpSecVpnHandlerProxyImpl.java:135)

Environment

VMware Cloud Director 10.6.1.1

VMware NSX-T 4.2.1.2

Cause

This issue can occur when the NSX-T module named "nsx-search" encounters a fault as it tries to fetch data. This results in no data being returned to Cloud Director.

Resolution

Restart the NSX Manager(s) in the impacted cluster.The restart will result in the nsx-search module being re-initialized.

Additional Information

Configure the IPsec VPN Site Connections for the NSX Data Center for vSphere Edge Gateway in the VMware Cloud Director Service Provider Admin Portal VMware Cloud Director Tenant Portal

Shut Down and Power On the NSX Manager Cluster