NSX Manager Upgrade failed, Relaunching the UI throws error : "Cannot fetch inventory details: Search service is currently unavailable"

Products

• VMware NSX

Issue/Introduction

This issue specifically impacts upgrades from NSX versions 3.2.1 or earlier, or 4.0.0, to versions 3.2.2 or later, or 4.0.1 or later.

System -> System Overview tab doesn't load the whole data and throws error : "Failed to get report - An unknown error has occurred".

• Following error is seen under /var/log/search/search-manager.log
  

  2025-##-##T##:##:##.891Z ERROR http-nio-127.0.0.1-7440-exec-28 SearchSyncManagerImpl 4217 - [nsx@6876 comp="nsx-manager" errorCode="MP60524" level="ERROR" reqId="54a18ccb-####-####-####-############" subcomp="manager" username="######@#####"] [QueryExecution: ConsistencyCheck] Search service is currently unavailable.

• Inventory logs shows the following 
  

  SearchHealthCheck SearchServiceWatchDog 31867 - [nsx@6876 comp="nsx-manager" errorCode="MP60524" level="ERROR" subcomp="cm-inventory"] Error
  pinging elasticsearch
  java.io.IOException: Connection reset by peer

Environment

VMware NSX-T Data Center versions prior to 3.2.2

VMware NSX 4.0.0

Cause

On upgrade, a complete reindexing of Elasticsearch is performed. As part of reindexing, all existing indices must first be deleted. In this case, there was a failure in the deletion of those indices within Elasticsearch due to a timeout error. As a result, reindexing does not complete and the search service is marked unavailable.

Resolution

This issue has been addressed and fixed in NSX versions 3.2.2 and 4.0.1.1  

NSX 3.2.2 Release Notes

NSX 4.0.1 Release Notes

Workaround :

In order to recover from this situation, run the below commands on all three Manager Nodes 

• restart service search
• restart service policy

Relaunch the NSX UI. 

Additional Information

Impact/Risks:
When the search service is unavailable, users may encounter a persistent error message on the UI, preventing them from viewing data. The error message will state "Search service is currently unavailable. Please restart by using the 'restart service search' command.