Error while attempting to get group membership in the VMware NSX-T 3.2 UI

Products

• VMware NSX

Issue/Introduction

Symptoms:

• You are running NSX-T 3.2.x
• You attempt to create an object within NSX, but get a connectivity error message.
• Similar entries to the below can be seen in /var/log/syslog in the NSX Manager:

nsxmgr01 NSX 20355 SYSTEM [nsx@6876 audit="true" comp="nsx-manager" level="INFO" subcomp="manager"] UserName:'username' ModuleName:'Policy' Operation:'GET@/api/v1/infra/domains/default/groups/test/members/ip-addresses' Operation status: 'failure' Error: Error while getting membership for the Group /infra/domains/default/groups/<groupname> from enforcement point /infra/sites/default/enforcement-points/default. Probable cause: Connectivity issue - INVALID_ARGUMENT: Invalid request. Either the request is null or object id is missing.

• Similar entries to the below can be seen in var/log/idps-reporting/idps.log 

WARN CorfuRuntime-0 CorfuRuntime 13154 Tried to get layout from <manager-ip>:9000 but failed with exception:

java.util.concurrent.ExecutionException: org.corfudb.runtime.exceptions.NetworkException: Disconnected [endpoint=<manager-ip>:9000]

Environment

VMware NSX-T Data Center

Cause

An exception, such as Network connectivity, can lead CCP (Central Control Plane) failing to re-connect to the UFO (Unified Framework Object) data store after the temporary disconnect, leading to CCP and UFO being in an un-synced state.

Resolution

This issue is resolved in VMware NSX-T 3.2.2.

 

Workaround:

A rolling restart of the CCP service on all the NSX managers, one at a time, is required to reestablish a connection between the CCP and the UFO store.
 

Steps to carry out the rolling restart:

1. As root user, confirm the service is running:
   • Command: #/etc/init.d/nsx-ccp status
2. Restart the service:
   • Command: #/etc/init.d/nsx-ccp restart
3. Validate the restart occured and the service is running again:
   • Command: #/etc/init.d/nsx-ccp status
4. Repeat steps 1 to 3 on the other 2 manager nodes.

Additional Information

NSX-T 3.2.0/3.2.0.1 VMs permanently lose network after vMotion or newly created VMs cannot connect