Unable to Add Standalone ESXi Host to vCenter Due to "SSL Handshake Failed" or "Host name does not match"
search cancel

Unable to Add Standalone ESXi Host to vCenter Due to "SSL Handshake Failed" or "Host name does not match"

book

Article ID: 426701

calendar_today

Updated On:

Products

VMware vSphere ESX 8.x VMware vCenter Server 8.0

Issue/Introduction

  • When adding a newly built or standalone ESXi host to the vCenter Server inventory, the process fails with the error
Unable to add the host
  • When checking the ESXi host certificate via SSH using openssl s_client -connect localhost:443, the output shows an "unstructuredName" in the Subject and an issuer of "VMware Installer
depth=0 C = US, ST = California, L = Palo Alto, O = "VMware, Inc", OU = VMware ESX Server Default Certificate, emailAddress = <UPN>@example.com, CN = <FQDN_host>, unstructuredName = "17#####99,5##d7###52049###32e"
verify error:num=20:unable to get local issuer certificate
depth=0 C = US, ST = California, L = Palo Alto, O = "VMware, Inc", OU = VMware ESX Server Default Certificate, emailAddress = <UPN>@example.com, CN = <FQDN_host>, unstructuredName = "17#####99,5##d7###52049###32e"
verify error:num=21:unable to verify the first certificate
0 s:C = US, ST = California, L = Palo Alto, O = "VMware, Inc", OU = VMware ESX Server Default Certificate, emailAddress = <UPN>@example.com, CN = <FQDN_host>, unstructuredName = "17#####99,5##d7###52049###32e"
   i:O = VMware Installer
  • The vCenter Server logs (/var/log/vmware/vpxd/vpxd.log) record the following SSL exceptions:
YYYY-MM-DDThh:mm:ss.Z warning vpxd[27875] [Originator@6876 sub=IO.Connection opID=mk3jarqc-######-auto-5fk7-h5:###2859-1c] Failed to SSL handshake; SSL(<io_obj p:0x00007f1d84386220, h:43, <TCP '<FQDN_OF_ESXIhost> : 46302'>, <TCP '<FQDN_OF_vCenter>> : 443'>>), e: 167772294(certificate verify failed (SSL routines)), duration: 20msec
YYYY-MM-DDThh:mm:ss.Z warning vpxd[27875] [Originator@6876 sub=HttpConnectionPool-000001 opID=mk3jarqc-#####-auto-5fk7-h5:###2859-1c] Failed to get pooled connection; <cs p:00007f1dc42a7a10, TCP:<FQDN_OF_ESXIhost>:443>, SSL(<io_obj p:0x00007f1d84386220, h:43, <TCP 'FQDN_OF_ESXIhost : 46302'>, <TCP '<FQDN_OF_vCenter> : 443'>>), duration: 37msec, N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: ##:0F:E0:##:BC:9F:##:0A:B1:##:81:DB:##:9C:B5:##:0E:1B:##:43
--> ExpectedThumbprint:
--> ExpectedPeerName: <FQDN_OF_ESXIhost>
--> The remote host certificate has these problems:
-->
--> * Host name does not match the subject name(s) in certificate.
-->
--> * unable to get local issuer certificate)

--> [context]zKq7AVECAQA#######nB4ZAAAowFTb######9yZS5zbwAA/gdCAB8vQw########QQDID0EAqRZBA3######3DcAFzU4ALv/UAGwjgB3#####yZWFkLnNvLjAAAu/6D2xp#####uNgA=[/context]

Cause

The ESXi installer sometimes generates a default self-signed certificate with an unstructuredName attribute that vCenter Server (specifically the vpxd service) cannot correctly parse or validate.

Because the Common Name (CN) or Subject Alternative Name (SAN) is not correctly formatted or the "VMware Installer" CA is not trusted by the vCenter Server's VMware Certificate Authority (VMCA), the SSL handshake fails during the host addition process.

Resolution

To resolve this issue, a new certificate must be generated for the ESXi host and signed by the vCenter Server's VMCA.

Step 1: Generate a CSR on the ESXi Host

  1. Log in to the ESXi host via SSH as root.

  2. Create or edit an openssl.cfg file in a temporary directory with the following content (replace placeholders with your actual environment details):

    [ req ]
default_bits = 2048
default_keyfile = rui.key
distinguished_name = req_distinguished_name
encrypt_key = no
prompt = no
string_mask = nombstr
req_extensions = v3_req

[ v3_req ]
basicConstraints = CA:FALSE
keyUsage = digitalSignature, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth, clientAuth
subjectAltName = DNS:<esxi_fqdn>, IP:<esxi_ip_address>, DNS:<esxi_shortname>

[ req_distinguished_name ]
countryName = <Country_Code>
stateOrProvinceName = <State>
localityName = <City>
0.organizationName = <Company_Name>
organizationalUnitName = <Department_Name>
commonName = <esxi_fqdn>
  3. Execute the command to create the crs file. 

    openssl req -new -nodes -out /tmp/rui.csr -keyout /tmp/rui.key -config openssl.cfg
  4. By using WinSCP, copy the rui.csr and rui.key  to local system

Step 2: Sign the Certificate on vCenter Server

  1. Copy the rui.csr from the ESXi host to the vCenter Server (e.g., to /storage/core/) using a tool like WinSCP.

  2. Log in to the vCenter Server via SSH as root.

  3. Run the following command to sign the CSR using the internal VMCA:

    openssl x509 -req -days 730 -in /storage/core/rui.csr -out /storage/core/rui.crt -CA /var/lib/vmware/vmca/root.cer -CAkey /var/lib/vmware/vmca/privatekey.pem -extensions v3_req -CAcreateserial

Step 3: Install the New Certificate on the ESXi Host

  1. Copy the newly created rui.crt and the original rui.key back to the ESXi host.

  2. Place them in the following directory: /etc/vmware/ssl/ (it is recommended to back up existing files first).

  3. Restart management services on the ESXi host:

    services.sh restart

Step 4: Re-add the Host

Return to the vSphere Client and add the ESXi host to the inventory.

 

Powered by Wolken Software