Error "An Unexpected Error Has Occurred. Please Try Again Later" when vIDM node health status fails to load

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• When attempting to fetch details of node 01 from vIDM UI > Administrator Portal > Dashboard > System Diagnostics Dashboard > expand details of specific Node : The details have a spinner running indefinitely, which then fails to load Port and ACS health status with error : "An Unexpected Error Has Occurred. Please Try Again Later".
• Horizon service and OpenSearch service logs have error once "Connection Time Out"
• If it is a single node, the database health may be marked as critical / Red. 

Environment

VMware Identity Manager 3.3.x

Cause

• The /etc/hosts file on the node contains external IP mapping for localhost, in addition to the localhost mapping default ip address 127.0.0.1, listed below. (This may see having impact only if the external IP address is listed above the default mapping for localhost, as first occurance take precedence when the /etc/hosts file is read for IP/fqdn resolution) 
• Many web APIs used for the health check (JDBC health check for example) do not allow external IP allocation, thus when the /etc/hosts file returns the external IP as a result of returning the first resolution found when reading the file from top to bottom, return the external Ip address which wouldn't be permitted and thus not able to retrieve the data.

Resolution

• Update the /etc/hosts file to remove / comment out the duplicate mapping for the localhost to external IP. 
• Restart the horizon-workspace service using command : /etc/init.d/horizon-workspace restart
• Validate that the node health status now reflects successfully. and the cluster health is green as expected.