Cluster Degraded Alarms Intermittently Open and Self-Resolve Due to Keystore Certificate Mismatches in NSX

Products

• VMware NSX

Issue/Introduction

In NSX UI, users may observe intermittent Cluster Degraded alarms that automatically open and self-resolve within the NSX Manager UI. During this period, all NSX Manager services display as UP, and the overall Manager cluster state remains STABLE.

Further investigation may reveal that these alarms appear to self-resolve without user intervention, and all system certificates show as VALID in the NSX UI.

Running the CARR script in dry run mode may display multiple keystore mismatches as shown below.

Environment

VMware NSX 4.x

Cause

These mismatches indicate underlying inconsistencies between NSX Manager keystores and their registered certificates, which cause temporary service flapping. This behavior triggers the transient Cluster Degraded alarms seen in the UI.

Resolution

Workaround:

1. Run the CARR Script
   Execute the CARR script without the -d (dry run) flag to remediate the keystore mismatches as outlined in Using Certificate Analyzer, Results and Recovery (CARR) Script to fix certificate related issues in NSX.

   ./start.sh

2. Confirm Fixes
   After validation completes, type "Yes" when prompted to apply the fixes to the mismatched keystore certificates.

3. Perform Rolling Reboot
   Once all mismatches are repaired and validations report no additional issues, perform a rolling reboot of the NSX Managers one at a time.

4. Validate Post-Reboot

   1. Ensure there are no further self-resolving Cluster Degraded alarms in the NSX UI.

   2. If any Cluster Unavailable alarms appear due to the manager reboots, manually select and Resolve them. These alarms should not return once the cluster is stable.