vSAN Skyline Health reports Failed to get KMS status due to CheckKmsStatusForDefaultView timeout

Products

• VMware vSAN

Issue/Introduction

In VMware vSAN environments configured with Data-at-Rest Encryption using a Standard Key Provider, the vSAN Skyline Health check VMware vCenter and all hosts are connected to Key Management Servers intermittently flaps between Green, Yellow, and Red status. The vSphere Client UI displays the issue Failed to get KMS status.

Reviewing /var/log/vmware/vsan-health/vmware-vsan-health-service.log on the vCenter Server Appliance reveals failures during key attribute lookups followed by health query timeouts:

 

ERROR vsan-mgmt [VsanVcEncryption::GetKmipKeyAttributes] Error reason: Failed to get key <ECRET> attributes from KMS <IP>:5696, reason: QLC_ERR_NONE
ERROR vsan-mgmt [VsanHealthEncUtil::GetKmipEncryptionKeyExpirationInfo] GetKmipKeyAttributes failed, please check you pass correct keyId and attribute names
INFO vsan-mgmt [VsanHealthEncUtil::GetClusterEncryptionConfig] RetrieveKmipServers result: running, Timed out
WARNING vsan-mgmt [VsanHealthEncUtil::_AggregateEncryptionConfigHealth] Host: HOSTNAME

Environment

• VMware vSAN (All Versions)

• KMS

Cause

This issue occurs when vCenter attempts to retrieve key attributes from one or more unreachable or unresponsive KMS provider endpoints. Because vCenter performs individual attribute queries against each configured KMS IP, an unresponsive node causes the health polling thread to exceed the default 15-second execution window, triggering a timeout alarm across the cluster.

Resolution

1. Log in to the vCenter Server Appliance (VCSA) via SSH.
2. Test connectivity to each configured KMS IP on port 5696: nc -z -v -w 5 <KMS_IP> 5696
3. Identify any endpoints returning a "Connection timed out" or "100% packet loss" via ping.
4. In the vSphere Client, navigate to vCenter Server > Configure > Key Providers.
5. Select the affected Key Provider and remove the unreachable KMS server endpoints.
6. Restart the vSAN health daemon to clear cached timeouts: vmon-cli -r vsan-health
7. Navigate to Cluster > Monitor > Skyline Health and click RE-RUN CHECKS.

Additional Information

vSAN Health Alarm - vCenter and all hosts are connected to Key Management Servers

For general KMIP port requirements, see VMware Ports and Protocols.

To speak with a customer representative or a Support Engineer see Contact Support. Scroll to the bottom of the page and click on your respective region.