Unable to search certain Users & Groups in the vIDM admin UI or assign Users & Groups to policies

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• Searching for groups within the vIDM admin console do not work
• The analytics health shows connected.
• Groups have been synced successfully within vIDM, but do not show up when searching
• The /opt/vmware/horizon/workspace/logs/analytics-service.log may show similar entries:

2020-04-08 22:39:26,175 GMT ERROR hostname:analytics (vert.x-eventloop-thread-0) [;;;] com.vmware.vidm.common.async.RetryCompletableFuture - Failed after max retries: 0 java.util.concurrent.CompletionException: com.vmware.vidm.common.http.client.HttpException: io.vertx.core.VertxException: Connection was closed
        at java.util.concurrent.CompletableFuture.encodeRelay(CompletableFuture.java:326)

Environment

VMware Identity Manager 3.3.x

Cause

Specific Users & Groups are not searchable typically due to issues with Elasticsearch/Opensearch and/or deleting unassigned shards

Resolution

1. Create snapshot on all vIDM nodes.
2. Locate the vIDM primary node
   • Log into the FQDN of the primary node directly (https://vIDM_PrimaryNode_FQDN/SAAS/login/0). Do not use the Load Balanced/VIP FQDN for the cluster
     • Select System domain and log in as admin
     • Using an alternative account such as configadmin will result in 403 forbidden errors. You must use the default admin account 
3. Use the developer tab in your browser to find the cookies (eg for Firefox, select Web Developer→ Network.  Then load a page and select the first request.  Then select the Cookies tab and scroll down to the HZN cookie).
   Copy the value of the HZN cookie.
4. Open a SSH session to the vIDM primary node and make the following REST API call, replacing <cookie_value> with the HZN cookie value obtained from the browser.

curl -k -XPUT -H "Authorization:HZN <cookie_value>" -H "Content-Type: application/vnd.vmware.horizon.manager.systemconfigparameter+json" https://localhost/SAAS/jersey/manager/api/system/config/SearchCalculatorMode -d '{ "name": "SearchCalculatorMode", "values": { "values": ["REINDEX"] } }'

The Users and Groups should now be searchable within the vIDM admin UI after a few minutes.