High CPU on NSX Manager when Directory Groups exceed 70,000 Entries

Products

• VMware NSX Data Center for vSphere

Issue/Introduction

• NSX Manager CPU stays over 89% constantly and sometimes reach 100%. 
• NSX Management Service stops and restarts Frequently
• vCenter Server Plugin doesn't load with "error message: Could not establish communication with NSX Manager. Please contact administrator"
• Rebooting the NSX Manager doesn't help with the issue.
• High CPU alerts in logs or graphs in UI
• In VSM log, we will observe NSX management service restarting and outofmemory logs:

2020-05-19 16:22:13.034 CEST INFO localhost-startStop-1 VsmServletContextListener:75 - - [nsxv@6876 comp="nsx-manager" subcomp="manager"] NSX Status : RUNNING
2020-05-19 16:31:40.550 CEST INFO localhost-startStop-2 VsmServletContextListener:75 - - [nsxv@6876 comp="nsx-manager" subcomp="manager"] NSX Status : STOPPED
2020-05-19 16:32:14.883 CEST INFO localhost-startStop-1 VsmServletContextListener:75 - - [nsxv@6876 comp="nsx-manager" subcomp="manager"] NSX Status : STARTING
 
2020-05-19 16:42:36.348 CEST ERROR TaskFrameworkExecutor-3 Worker:227 - - [nsxv@6876 comp="nsx-manager" subcomp="manager"] Exception thrown while executing task instance taskinstance-3094799
java.lang.OutOfMemoryError: GC overhead limit exceeded

• In NSX DB, DirectoryGroup entries are shown to be exceeding the limits of 70K.

Environment

VMware NSX Data Center for vSphere 6.4.x

Cause

This is a Design issue, Maximum Directory Group Entries supported is 70,000 Entries.
Please refer to the configuration Max tool under the Identity Firewall (VMware Configuration Maximums)

Exceeding above limit will cause is CPU utilization to increase

Resolution

To Completely resolve this issue, NSX Manager should be Integrated with an external domain with Active Directory Groups entries less than 70k.

Workaround:
Please raise a support request if you are encountering this issue.

Additional Information

Impact/Risks:
Changes not getting pushed via NSX Plugin UI
NSX Plugin UI becomes unresponsive.