Alarms on ESXi host: "Few Hosts Requires Encryption Mode Enabled"

Products

• VMware vSphere ESXi

Issue/Introduction

• Getting an alarm on ESXi host as below:
  

  The issue is caused by vCenter and ESXi hosts not being able to establish a trusted connection with the external CloudLink Key Management Server (KMS).

  
  

  Host <host FQDN> requires encryption mode enabled. Check the status of the key provider <External KMS> and manually recover the missing key #############-####-####-############/to the key provider <External KMS>

• While enabling the encryption in the location - ESXi host --> Configure --> Security Profile --> Enable Encryption we get the below error message:

  A general runtime error occurred.
  Cannot generate key. CreateKey failed on key provider <External KMS>, error code:QLC_ERR_COMMUNICATE;
  Failed. Check log for details.

• In the vCenter logs on location: /var/log/vmware/vpxd/vpxd.log we get below error message:

  warning vpxd[191700] [Originator@6876 sub=VpxProfiler opID=WorkQueue-7e9dcfa3] WorkQueue [TotalTime] took 2031 ms error vpxd[191831] [Originator@6876 sub=CryptoManagerKmipWrapper opID=m800ek6j-45696088-auto-r7fbt-h5:######-71] Failed to connect to key server <External KMS IP>:5696 - Err:QLC_ERR_COMMUNICATE Failed to establish the connection: Q_ERROR_FAILED --> error vpxd[191831] [Originator@6876 sub=CryptoManagerKmipWrapper opID=m800ek6j-#######-auto-r7fbt-h5:7######4-71] Failed to connect to key server <External KMS IP>:5696 - Err:QLC_ERR_COMMUNICATE Failed to establish the connection: Q_ERROR_FAILED -->  warning vpxd[191831] [Originator@6876 sub=Default opID=m800ek6j-4#######-auto-r7fbt-h5:#########-71] Failed to get key <Encryption Key>on key provider, error 2:
  
  --> Reason: --> Failed to get key <Encryption Key> on KMS <KMS IP>: QLC_ERR_COMMUNICATE; --> Failed to get key <Encryption Key>on KMS <KMS IP>: QLC_ERR_COMMUNICATE error vpxd[191831] [Originator@6876 sub=CryptoManagerKmipWrapper opID=m800ek6j-4#######-auto-r7fbt-h5:######84-71] Failed to connect to key server 1<KMS IP>:5696 - Err:QLC_ERR_COMMUNICATE Failed to establish the connection: Q_ERROR_FAILED.

• KMS server is not trusted on the vCenter.

Environment

VMware vCenter Server 8.x

Cause

The issue is caused by vCenter and ESXi hosts not being able to establish a trusted connection with the external Key Management Server (KMS).

Resolution

Trust must be established between vCenter and the remote KMS server after replacing certificates in a vSphere environment that is configured to use a KMS Server.

We can use the KB article to establish the trust: Re-establishing Trust between vCenter and KMS after certificate replaement.