"Host Requires Encryption Mode Enabled" alarm/event triggered on ESXi hosts with no Encryption

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

• Following alarm is triggered on the ESXi:
  
  
  OR
  
  
• The host keeps getting the 'Host Requires Encryption Mode Enabled. Check the status of the key provider NKP and manually recover the missing key' events in the monitor tab.

• Encryption is not enable on the ESXi host.
• Secure Boot is disabled.
• The ESXi host is not part of a mixed cluster where some hosts have TPM enabled.
• A Key Management Server (KMS) is not configured in the environment.

 

Environment

vSphere ESXi
vCenter Server

Cause

The issue is seen when the encryption is not enabled on cluster and hosts. However, encryption was configured and disabled which would have left the stale entries in the vCenter database and failed to clean up.

On the vCenter Database, we would see encryption details for the hosts as shown below:

VCDB=# select id, dns_name, crypto_state, crypto_key_id, crypto_key_provider_id, crypto_enable from vpx_host where dns_name like '%HOST_FQDN%';
 id |          dns_name          | crypto_state |                                            crypto_key_id                                                      |          crypto_key_provider_id        |        crypto_enable
----+----------------------------+--------------+----------------------------------------------------------------------------------------------------------------------------------------------+------------------------+-------------
 41 |         HOSTNAME           |            0 |                                      Stale_Crypto_Key_ID_example                                              |               Old_NKP_Name             |             1
(1 row)

The above output show that vCenter has the encryption information for the host which was not cleaned properly. This could be due to a mismatch of the information between the ESXi Host and the vCenter. The vCenter information points to the ESXi host still having the Host Key available, but the host does not have access to the key. 

Resolution

In order to fix the issue, remove the stale entries from the vCenter database and then verify the details.

1. Verify host level encryption information using the below command:

esxcli system settings encryption get

When the encryption is not enabled, the output for the above command will give the below result.

esxcli system settings encryption get
   Mode: TPM
   Require Executables Only From Installed VIBs: false
   Require Secure Boot: False

Once it is confirmed there is no encryption enabled on the hosts and vCenter, we can proceed to clean up the stale entries in vCenter server VCDB.

2.  Important: Before making any changes to the vCenter database, ensure you have a valid backup and snapshot of the vCenter Server. For vCenter in ELM :  VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practiceT

3. Below steps need to be performed to fix the issue:

     1. Stop the vpxd service on the vCenter by using below command:

service-control --stop vmware-vpxd

     2. Connect to the VCDB via the postgres shell using below command:

/opt/vmware/vpostgres/current/bin/psql -d VCDB -U postgres

     3. Identify the information for the host in question:

select id, dns_name, crypto_state, crypto_key_id, crypto_key_provider_id, crypto_enable from vpx_host where dns_name like '%esxi_FQDN%';

     4. Using the "id" for the affected host, run the following command to clear its stale information

update vpx_host set crypto_state=0, crypto_enable=0, crypto_key_id='', crypto_key_provider_id='' where id=<host_id>;

Note: Replace <host_id> with the "id" of the affected host

     5. Quit the postgres shell and restart vpxd:

\q
service-control --start vmware-vpxd 

Verify the output of the command again and it should not show any key related information as below: