After upgrading to VMware NSX 4.1.1, (Error code: 60516) is observed in the NSX UI

Products

• VMware NSX

Issue/Introduction

Symptoms:

• You've recently upgraded to VMware NSX version 4.1.1, and removed the Advanced Load Balancer (ALB) configuration from the NSX UI.
• The Networking part in the NSX UI stops working with the error: INDEXING_FAILURES_EXHAUSTED_RETRIES, params: [all] (Error code: 60516).
• You've attempted the fix proposed in the error: "Run the 'start search resync all' CLI command on the NSX appliance to resync", and the error still exists. 
• In the /var/log/search/search-manager.logs file on an NSX-T manager, you see entries similar to:

2023-08-22T19:25:48.935Z  WARN pool-742-thread-1 GenericPolicyRealizedResourceDataProvider 105456 - [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] [IndexDataProvider-GenericPolicyRealizedResource] Could not fetch data to be indexed for GenericPolicyRealizedResource//infra/realized-state/enforcement-points/alb-endpoint/alb-virtual-services/portainer,
cause : 
com.vmware.nsx.management.policy.policyframework.exceptions.InvalidPolicyPathException:
The path=[/infra/realized-state/enforcement-points/alb-endpoint/alb-virtual-services/portainer] is invalid 

• You may also see similar entries to the below in the /var/log/proton/nsxapi.log

type.googleapis.com/vmware.nsx.ufostore.advanceloadbalancer.model.AlbControllerAdminCredsMsg not in map!"

Environment

VMware NSX-T Data Center

Cause

This issue occurs when some ALB objects remain in NSX-T DB.
From VMware NSX 4.1.1 onwards, some ALB integration features were removed, this lead to some stale objects remaining and resulted in the above Generic Policy Realized Resource indexing Alarm above: InvalidPolicyPathException.
Please refer to the following KB for more detail on the ALB deprecation: Deprecation Announcement of NSX-T ALB Policy API and UI.

Resolution

This issue is resolved in VMware NSX 4.1.2

Workaround:

• If you believe you have encountered this issue, please open a support request with Broadcom Support and refer to this KB article.

Additional Information

Impact/Risks:

The Networking part in the NSX UI stops working, which will prevent the user from viewing the tier1, segments and the tier0.