Cannot fetch inventory details: Search service is currently unavailable

Products

• VMware NSX
• VMware NSX-T Data Center

Issue/Introduction

Global Manager (GM) UI is reporting the error message below after upgrading the Local Manager (LM). This error is only observed in the GM when selecting the LM that got upgraded. It does not exist if logging in to the LM directly.

[nsx@6876 comp="nsx-manager" errorCode="PM513002" level="ERROR" subcomp="manager"] Cannot fetch inventory details: Search service is currently unavailable, please restart using 'restart service search' and resync using 'start search resync all' CLI commands on the NSX appliance.
~
[nsx@6876 comp="nsx-manager" errorCode="MP60503" level="ERROR" subcomp="manager"] [Search: Initialization] Error while updating indexing metadata with id policy_topology
net.jodah.failsafe.FailsafeException: java.net.SocketTimeoutException: 30,000 milliseconds timeout on connection http-outgoing-24 [ACTIVE]
        at net.jodah.failsafe.SyncFailsafe.call(SyncFailsafe.java:159) ~[?:?]
        at net.jodah.failsafe.SyncFailsafe.run(SyncFailsafe.java:81) ~[?:?]
        at com.vmware.nsx.management.search.helpers.IndexingMetadataHelper.updateMetadata(IndexingMetadataHelper.java:91) ~[?:?]
        at com.vmware.nsx.management.search.helpers.IndexingMetadataHelper.updateMetadata(IndexingMetadataHelper.java:76) ~[?:?]
        at com.vmware.nsx.management.search.topology.helper.TopologyIndexingHelper.updateMetadata(TopologyIndexingHelper.java:54) ~

Environment

VMware NSX-T Data Center
VMware NSX

Cause

Issue is due to Elasticsearch template not being applied on search metadata indices and index position indices. This lead to errors while trying to read from metadata indices during search initialization which has caused the initialization failure on search.

Resolution

The issue for template not being applied on metadata indices has been resolved in 3.2.2 and 4.0.1. Users may continue to receive this error until all GM and LM are upgraded to 3.2.2 and 4.0.1.

Workaround:
Run 'restart service search' and resync using 'start search resync all' CLI commands on each NSX Local Manager appliance.

NSX Local Manager may return an error message "% An error occurred while re-syncing search service" after issuing the command  'start search resync all'. Please allow a few minutes for the search service to fully restart and stabilize before initializing the resync operation.