After upgrading to VMware NSX 4.1.X, (Error code: 60516) is observed in the NSX UI

Products

• VMware NSX

Issue/Introduction

• NSX version 4.1.x is upgraded and ALB configuration is removed from NSX UI
• The Networking tab in the NSX UI stops working with the error:

    INDEXING_FAILURES_EXHAUSTED_RETRIES, params: [all] (Error code: 60516).

• Attemping to resync with the command "start search resync all" on NSX CLI does not help
• In the /var/log/search/search-manager.logs file on an NSX-T manager, you see entries similar to:

2023-08-22T19:25:48.935Z  WARN pool-742-thread-1 GenericPolicyRealizedResourceDataProvider 105456 - [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] [IndexDataProvider-GenericPolicyRealizedResource] Could not fetch data to be indexed for GenericPolicyRealizedResource//infra/realized-state/enforcement-points/alb-endpoint/alb-virtual-services/portainer,
cause : 
com.vmware.nsx.management.policy.policyframework.exceptions.InvalidPolicyPathException:
The path=[/infra/realized-state/enforcement-points/alb-endpoint/alb-virtual-services/portainer] is invalid 

• You may also see similar entries to the below in the /var/log/proton/nsxapi.log

type.googleapis.com/vmware.nsx.ufostore.advanceloadbalancer.model.AlbControllerAdminCredsMsg not in map!"

Environment

VMware NSX-T Data Center

Cause

• This issue occurs when some ALB objects remain in NSX-T DB.
• From VMware NSX 4.1.1 onwards, some ALB integration features were removed, this lead to some stale objects remaining and resulted in the above Generic Policy
• Please refer to the following KB for more detail on the ALB deprecation: Deprecation Announcement of NSX-T ALB Policy API and UI.

Resolution

If you have encountered this issue, please open a support case with Broadcom Support, refer to this KB article: Creating and managing Broadcom support cases