NSX Manager GUI search fails with an error - "Error: Search index is out of sync. Run the 'start search resync all'

Products

• VMware NSX

Issue/Introduction

• Service config resource refers to Logical Router UUID, which no longer exists in the system. This causes exceptions in the UI search.
  
  "Error: Search index is out of sync. Run the 'start search resync all' CLI command on the NSX appliance to resync. If the issue persists, contact VMware Support. INDEXING_FAILURES_EXHAUSTED_RETRIES, params: [all] (Error code: 60516)"
  
  

Environment

VMware NSX-T Data Center
VMware NSX

Cause

Global search returns error "Index out of sync, please resync." This is due to missing data in ServiceConfig table for associated logical routers. Search resync doesn't help because there is missing data in corfu which prevents DTO convertion for ServiceConfig entries.

It is caused under 2 conditions:

1. Create a T1/T0 logical router and attach flood-protection profile to it. Delete T1/T0. This leaves flood-protection profile as stale meaning which a T1/T0 logical router was deleted sometime back and has never been created back with same T1/T0 name. Flood-protection profile will reference to old UUID. 
2. Create a T1/T0 logical router and attach flood-protection profile to it. Delete T1/T0. We now recreate T1/T0 logical router with the same T1/T0 name. This also leaves flood-protection profile as stale. Flood-protection profile will reference to old UUID. 

2024-03-29T19:59:33.782Z ERROR UfoIndexer-BatchExecutor-search_manager-1 UfoGenericConverter 4792 - [nsx@6876 comp="nsx-manager" errorCode="MP60511" level="ERROR" subcomp="manager"] [Indexing: DtoConversion] Could not convert UFO object to Dto by DTO converter UfoObject{operationType=CREATE, descriptor=IndexingTypeDescriptor{tableName='ServiceConfig', streamTag=MANAGER}, identifier=uuid {
 left: XXXXXXXXXXXXXXXXXXXX
 right: YYYYYYYYYYYYYYYYYYYY
}
}
Caused by: com.vmware.nsx.management.common.exceptions.ObjectNotFoundException
        at com.vmware.nsx.management.container.dao.IdentifiableProxyObjectDao.findByIdentifier(IdentifiableProxyObjectDao.java:400) ~[?:?]
        at com.vmware.nsx.management.edge.lrouter.service.LogicalRouterServiceImpl.getLogicalRouter(LogicalRouterServiceImpl.java:391) ~[?:?]
        at com.vmware.nsx.management.serviceconfig.converter.ServiceConfigConverter.toDtoWithTx(ServiceConfigConverter.java:219) ~[?:?]
        at com.vmware.nsx.management.serviceconfig.converter.ServiceConfigConverter.toDtoWithTx(ServiceConfigConverter.java:63) ~[?:?]
        ... 20 more

Resolution

 

This issue is resolved in VMware NSX 4.2.0

Workaround :
Delete the flood-protection-profile-bindings and recreate the same under that tier0/tier1 logical router.