After upgrading to vSphere 5, you see the HA error: vSphere HA Cannot be configured on this host because its SSL thumbprint has not been verified

Products

VMware

Issue/Introduction

Symptoms:

After upgrading to vSphere 5.0, you see one of these errors in your hosts:
- vSphere HA agent for this host has an error: The vSphere HA agent is not reachable from vCenter Server
- vSphere HA cannot be configured on this host because it's SSL thumbprint has not been verified. Check that vCenter server is configured to verify SSL thumbprints and that the thumbprint for this host has been verified
- There was an error unconfiguring the vSphere HA agent on this host. To solve this problem, connect the host to a vCenter Server of version 5.0 or later
vCenter Server is configured to verify SSL thumbprint for all hosts
Removing and re-adding the host to the cluster does not resolve the issue
Disabling and re-enabling HA does not resolve the issue

Cause

This issue occurs if custom SSL certificates were used with the hosts before the upgrade. The vCenter Server SQL database is unable to update the SSL thumbprint in the database.

Note: To compare the SSL thumbprint follow the steps given in Additional Information section below.

Resolution

Note: This issue is resolved in vCenter Server 5.0 Update 1. For more information, see the Resolved Section in the VMware vCenter Server 5.0 Update 1 Release Notes.

If the issue is still occurring after you upgrade to vCenter Server 5.0 Update 1, it may be necessary to clear the ssl state in your browser. For example in Internet Explorer, select Internet options > Content tab > Clear SSL state

Note: If you are encountering this issue and you are using SDK or vSphere Management Assistant (vMA), see the Alternative workaround.

To work around this issue on an earlier vCenter Server 5.0 release:

Open the instance.cfg file using a text editor. By default, the file is located at:
- Windows 2003: C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\
- Windows 2008: C:\ProgramData\VMware\VMware VirtualCenter\
Add this line at the end of the file:

keystorePassword=testpassword

Where testpassword is the password that was used to create the .pfx file. By default, it is testpassword .
Restart the VMware vCenter Server service, the VMware vCenter Management Webservices, and the VMware VCMSDS services. For more information, see Stopping, starting, or restarting vCenter services (1003895).
Remove the host from vCenter Server, then re-add it to vCenter Server.
Verify that vCenter Server is configured to verify SSL thumbprints using the vCenter Server Settings option.
If you select the vCenter requires verified host SSL certificate option, it will disconnect the hosts from vCenter unless all the hosts listed at the bottom are selected for SSL verification as well.
- Click Administration > vCenter Server Settings... > SSL Settings
  
  Note: You will see a list of hosts and have the option to select the Verified option for the list of hosts. Alternatively, you can select which hosts should be verified by selecting each one separately. Also ensure that the vCenter requires verified host SSL certificate option at the top is selected at the same time. Selecting only the vCenter requires verified host SSL certificate option disconnects the host from vCenter Server. Be sure to validate the host at the bottom by selecting the Verified option.

If the issue persists, try these steps:

Note: Ensure that you create a backup of the database before proceeding.

Shut down the VMware vCenter Server service.
Run these SQL statements on the vCenter Server database:

SELECT id,EXPECTED_SSL_THUMBPRINT,HOST_SSL_THUMBPRINT FROM dbo.VPX_HOST

Note: Take the results from the SELECT statement above, and use them to fill in the thumbprint and host ID values in the UPDATE statement below.

UPDATE dbo.VPX_HOST SET EXPECTED_SSL_THUMBPRINT = 'Insert Thumbprint here' WHERE id = 'host ID'

Alternatively, if you know the host ID, you can use this SQL statement:

SELECT HOST_SSL_THUMBPRINT into EXPECTED_SSL_THUMBPRINT from dbo.VPX_host WHERE id = 'host ID';
Start the VMware VirtualCenter Server service.
Reconfigure HA on all hosts.

Alternative workaround

To avoid having to restart the VirtualCenter Server service every time you add an ESX host, you can use the SDK script HostReconnect.pl . This script goes through all hosts and calls HostSystem.reconnect , passing the expected SSL thumbprint in ConnectSpec . This populates the EXPECTED_SSL_THUMBPRINT column in the database and does not require you to restart the VirtualCenter Server service.

Note: You must have vMA or SDK installed to run this script.

To run the script:

Download and extract FinalHostReconnect.rar , which is attached to this article. It contains the SDK script HostReconnect.pl .
Run this command from vMA or SDK:

perl HostReconnect.pl --server VC-server-IP --username usernameToConnectToVC
When prompted, enter your password to connect to vCenter Server.
For each host being added after the script is run, right-click the host in vCenter Server and select Reconfigure for vSphere HA. This successfully reconfigures HA on the host.

Caution: This script reconnects all hosts in the cluster. If some hosts are in a disconnected state before the script runs, they are reconnected.

If you have any questions about this script, file a support request with VMware Technical Support. For more information, see:

Additional Information

Additional commands to compare the thumbprint:

Open a browser and connect to an affected ESXi host, view the SSL certificate on the host, and record the thumbprint of the certificate.
OR,
Open an client to a Linux server / workstation with openssl, and run the following:

openssl s_client -connect ESXi_FQDN:443 </dev/null 2>/dev/null | openssl x509 -fingerprint -noout -in /dev/stdin

Note: Where ESXi_FQDN is the ESXi FQDN of an affected ESXi host
Log into SQL Server Management Studio, and run the following query against the vCenter Server database:

select ID,DNS_NAME,EXPECTED_SSL_THUMBPRINT,HOST_SSL_THUMBPRINT from VPX_HOST;

Compare the output of step 1 and 2 from above. If they are different, the issue lies theere.

How to stop, start, or restart vCenter Server services
Configuring HA after upgrading to vCenter Server 5.0 fails with the error: Cannot complete the configuration of the vSphere HA agent on the host. Misconfiguration in the host setup
How to file a Support Request in Customer Connect
vSphere 5 にアップグレードした後、HA エラー「SSL サムプリントが検証されていないため、このホストに vSphere HA を構成できません (vSphere HA Cannot be configured on this host because its SSL thumbprint has not been verified)」が表示される
升级到 vSphere 5 后，会显示 HA 错误：无法在该主机上配置 vSphere HA，因为尚未验证其 SSL 指纹 (vSphere HA Cannot be configured on this host because its SSL thumbprint has not been verified)

Attachments

FinalHostReconnect.rar get_app