NSX Upgrade from SDDC failing with Health Check failures for cluster

Products

• VMware SDDC Manager

Issue/Introduction

During the NSX Upgrade from the SDDC Manager, the Upgrade - NSX_HOSTCLUSTER task fails.

From the vSphere, the Apply NSX Solution task fails with the following error:

A general system error occurred: Health Check for '<ClusterName>' failed.

From the NSX Manager, the following error is displayed:

Upgrade failed: Fault Domain Manager vib is missing on some of the hosts in the cluster '<ClusterName>'. Ensure the vib is installed on all hosts or try re-enabling vSphere HA on the cluster.

Environment

SDDC Manager 5.2.x

Cause

This is caused by some hosts in the cluster not having the correct vmware-fdm vib installed.

This can be confirmed by running the following command on each host in the cluster:

esxcli software vib list | grep -i fdm

This will list the "vmware-fdm" vib and its version information, if present. 

Also, in the vSphere UI, Select the cluster. Then select the 'Hosts' tab to see if there are any hosts that are not configured for HA.

Resolution

In order to resolve this issue, disable and re-enable HA for the cluster. This will install the vib on any hosts that does not have it and upgrade the vib on hosts that have an older version.

Instructions on this process can be found here:

Disabling and enabling VMware vSphere High Availability (vSphere HA)