Host requires encryption mode enabled alert is triggered as the KMS encryption server certificate was expired.

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

• The host requires an encryption mode-enabled alert to be triggered, and the encryption on the host is disabled.
  
  
• When you SSH to the host and run the following command:

[root@esxi-host:~] crypto-util keys getkidbyname "HostKey"
crypto-util keys: A key for 'HostKey' has not been established.

• From the vCenter's /var/log/vmware/vpxd logs, we observe the following snippets:

YYYY-MM-DDTHH:MM:SS error vpxd[07646] [Originator@6876 sub=CryptoManagerKmipWrapper opID=m2td2ofk-4162346-auto-2h7os-h5:70273839-44-01-01-SWI-39c153fc] Failed to connect to key server ip_address_of_KMS_server:port
 - Err:QLC_ERR_COMMUNICATE
-->
YYYY-MM-DDTHH:MM:SS warning vpxd[07646] [Originator@6876 sub=Default opID=m2td2ofk-4162346-auto-2h7os-h5:70273839-44-01-01-SWI-39c153fc] Failed to generate key on key provider key_provider_name, error 2:
--> Reason:
--> Failed to generate key on KMS ip_address_of_KMS_server: QLC_ERR_COMMUNICATE
--> Custom attribites: (null)
-->
YYYY-MM-DDTHH:MM:SS error vpxd[07646] [Originator@6876 sub=CryptoManager opID=m2td2ofk-4162346-auto-2h7os-h5:70273839-44-01-01-SWI-39c153fc] Failed to enable encryption on [vim.HostSystem:host-*****,name_of_host_in_question]: N5Vmomi12RuntimeFault9ExceptionE(Fault cause: vmodl.RuntimeFault
--> )
--> [context]zKq7AVECAQAAAOIfcwEXdnB4ZAAAAto3bGlidm1hY29yZS5zbwAAmXksABdtLQAf6jIBvRoObGlidm1vbWkuc28AAeNtDAKXHXN2cHhkAILD2h8BgtQfHAGCvyAcAYKEPxwBgjFHHAGCtUocAYKXuBwBgke+HAGCphFmAYJAEmYBAJCzIwDnSSMAdZ8jAMBlNwOHfwBsaWJwdGhyZWFkLnNvLjAABL82D2xpYmMuc28uNgA=[/context]
YYYY-MM-DDTHH:MM:SS error vpxd[07646] [Originator@6876 sub=CryptoManager opID=m2td2ofk-4162346-auto-2h7os-h5:70273839-44-01-01-SWI-39c153fc] Failed to enable encryption on [vim.HostSystem:host-*****,name_of_host_in_question]: N5Vmomi5Fault12NotSupported9ExceptionE(Fault cause: vmodl.fault.NotSupported
--> )
--> [context]zKq7AVECAQAAAOIfcwETdnB4ZAAAAto3bGlidm1hY29yZS5zbwAAmXksABdtLQAf6jIBa5gYbGlidm1vbWkuc28AAatxDAKXHXN2cHhkAII9SRwBgrVKHAGCl7gcAYJHvhwBgqYRZgGCQBJmAQCQsyMA50kjAHWfIwDAZTcDh38AbGlicHRocmVhZC5zby4wAAS/Ng9saWJjLnNvLjYA[/context].
YYYY-MM-DDTHH:MM:SS error vpxd[07646] [Originator@6876 sub=CryptoManager opID=m2td2ofk-4162346-auto-2h7os-h5:70273839-44-01-01-SWI-39c153fc] Failed to recover [vim.HostSystem:host-*****,name_of_host_in_question] crypto state from "incapable" to "safe" in [vim.ClusterComputeResource:domain-*****,name_of_cluster_which_host_in_question_is_part_of].
YYYY-MM-DDTHH:MM:SS info vpxd[06887] [Originator@6876 sub=CryptoManager opID=q-88637:h5ui-getProperties:urn:vmomi:Folder:group-d1:9e519b2b-cd3f-46eb-998b-11cfc71af547:1512692352:VCenterKmipPropertyProvider:380453-2jt1t-h5:70287165-25] Check certificate expiry from cluster key_provider_name
YYYY-MM-DDTHH:MM:SS warning vpxd[06887] [Originator@6876 sub=CryptoManager opID=q-88637:h5ui-getProperties:urn:vmomi:Folder:group-d1:9e519b2b-cd3f-46eb-998b-11cfc71af547:1512692352:VCenterKmipPropertyProvider:380453-2jt1t-h5:70287165-25] Certificate [Subject: UID=*****,emailAddress=*****,CN=*****,O=*****] expires on YYYY-MM-DD HH:MM:SS
YYYY-MM-DDTHH:MM:SS warning vpxd[06887] [Originator@6876 sub=CryptoManager opID=q-88637:h5ui-getProperties:urn:vmomi:Folder:group-d1:9e519b2b-cd3f-46eb-998b-11cfc71af547:1512692352:VCenterKmipPropertyProvider:380453-2jt1t-h5:70287165-25] KMS Client certificate expires on YYYY-MM-DDTHH:MM:SS will generate an alarm
YYYY-MM-DDTHH:MM:SS error vpxd[06971] [Originator@6876 sub=Main opID=CheckCertificateExpiry-6c4f3258] Unable to get certificates from the store APPLMGMT_PASSWORD
YYYY-MM-DDTHH:MM:SS warning vpxd[06971] [Originator@6876 sub=Main opID=CheckCertificateExpiry-6c4f3258] Certificate [Subject: UID=*****,emailAddress=*****,CN=*****,O=*****] from store KMS_ENCRYPTION will expire on YYYY-MM-DDTHH:MM:SS

 

Environment

VMware vCenter Server 7.x
VMware vCenter Server 8.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

The certificate of the KMS server is expired.

Resolution

1. Ensure the Host is in Maintenance mode.
2. Renew the certificate of the KMS encryption server.
3. Enable encryption on the ESXi host. Refer: Re-Activate ESXi Host Encryption Mode 
4. Exit the host from the Maintenance mode.
5. Click on "Reset to green" of the alert appearing for the host in question, where the encryption was enabled in the previous step.