"Error: Search index is out of sync for 'IP Set, MACSet, NSGroup'" when reaching NSX Local Manager's alarms page in NSX Federation environment

Products

VMware NSX

Issue/Introduction

Following an upgrade from NSX 3.x to NSX 4.x, the NSX UI's Alarm page is no longer loading
The following message is displayed when attempting to reach the NSX Alarm page:

Error: Search index is out of sync for 'IP Set, MACSet, NSGroup'. Run the 'start search resync all' CLI command on the NSX appliance to resync. If the issue persists, contact VMware Support. (Error code: 60516)
In the VMware NSX Search Manager logs (/var/log/search/search-manager.log) the following errors can be seen, please note that the number of batch items (groups) that may have the problem


2025-10-14T12:23:40.408Z ERROR http-nio-127.0.0.1-7440-exec-3 SearchSyncManagerImpl 6174 - [nsx@6876 comp="nsx-manager" errorCode="MP60516" level="ERROR" reqId="######################" subcomp="manager" username="######"] [QueryExecution: ConsistencyCheck] Search index is out of sync for IPSet, MACSet, NSGroup.

2025-10-13T14:24:44.609Z  INFO UfoIndexer-search_manager-0 UfoIndexer 6260 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] [Indexing] 1 chunks are created out of 15 batch items
2025-10-13T14:24:44.580Z ERROR UfoIndexer-BatchExecutor-search_manager-5 UfoGenericConverter 6260 - [nsx@6876 comp="nsx-manager" errorCode="MP60511" level="ERROR" subcomp="manager"] [Indexing: DataProvider] Could not convert UFO object to Dto UfoObject{operationType=INDEX, descriptor=IndexingTypeDescriptor{tableName='InternalGroup', streamTag=MANAGER}, identifier=uuid {
  left:######################
  right:######################
}
}
java.lang.reflect.InvocationTargetException: null
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) ~[?:?]
        at java.lang.reflect.Method.invoke(Unknown Source) ~[?:?]
        at com.vmware.nsx.management.search.provider.UfoGenericConverter.internalConvert(UfoGenericConverter.java:102) ~[?:?]
        at com.vmware.nsx.management.search.provider.UfoGenericConverter.convertToDataToIndex(UfoGenericConverter.java:71) ~[?:?]
        at com.vmware.nsx.management.search.service.impl.UfoIndexingServiceImpl.processForIndexing(UfoIndexingServiceImpl.java:622) ~[?:?]
        at com.vmware.nsx.management.search.service.impl.UfoIndexingServiceImpl.processUfoObjectForIndexing(UfoIndexingServiceImpl.java:544) ~[?:?]
        at com.vmware.nsx.management.search.service.impl.UfoIndexingServiceImpl$1.process(UfoIndexingServiceImpl.java:513) ~[?:?]
        at com.vmware.nsx.management.search.configuration.DataStore.processDynamicTxBatch(DataStore.java:121) ~[?:?]
        at com.vmware.nsx.management.search.service.impl.UfoIndexingServiceImpl.processBatchForIndexing(UfoIndexingServiceImpl.java:501) ~[?:?]
        at com.vmware.nsx.management.search.manager.UfoIndexer.processChunk(UfoIndexer.java:260) ~[?:?]
        at com.vmware.nsx.management.search.manager.UfoIndexer.lambda$processQueue$1(UfoIndexer.java:183) ~[?:?]
        at java.util.concurrent.FutureTask.run(Unknown Source) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[?:?]
        at java.lang.Thread.run(Unknown Source) ~[?:?]
Caused by: java.lang.IllegalArgumentException: No enum constant com.vmware.nsx.management.policy.policyframework.model.GroupMemberType.INVALID ----> Error

From the corfu tables (InternalGroup and Group) of the affected NSX managers there will be static member list that is having an empty entry while the intent "Group" table was having group definition with segment path

/opt/vmware/bin/corfu_tool_runner.py -t Group -n nsx -o showTable >/image/Group.txt
/opt/vmware/bin/corfu_tool_runner.py -t InternalGroup -n nsx -o showTable >/image/InternalGroup.txt

Using REST API call:

GET https://{{mp_server}}/api/v1/ns-groups/
GET https://{{mp_server}}/policy/api/v1/infra/domains/default/groups/

Environment

VMware NSX Federation

Cause

There is inconsistent data between 'Management Plane' and 'Policy' Databases. The internal static membership inconsistency prevents services from running correctly, which causes parts of the NSX UI to not load properly and any searches in the UI to fail with an error message.
The static member list is having an empty entry while the intent "Group" table was having group definition with segment path. Due to this, internal conversion of framework data was failing as empty member is invalid, which in turn is impacting search indexing and the UI.

Resolution

If you believe you have encountered this issue, please open a support case with Broadcom Support and refer to this KB article.
For more information, see Creating and managing Broadcom support cases.

Additional Information

If you are contacting Broadcom support about this issue, please provide the following:

Please run the below corfu commands from the local manager that is having issues accessing alarms page:

/opt/vmware/bin/corfu_tool_runner.py -t Group -n nsx -o showTable >/image/Group.txt /opt/vmware/bin/corfu_tool_runner.py -t InternalGroup -n nsx -o showTable >/image/InternalGroup.txt

Using REST API call:

GET https://{{mp_server}}/api/v1/ns-groups/
GET https://{{mp_server}}/policy/api/v1/infra/domains/default/groups/

Please see below kb's for generating NSX logs bundles for offline review with Broadcom support