NSX Upgrade Failure: "Failed to install software on host" due to Expired ESXi Certificate

Products

• VMware NSX

Issue/Introduction

• During the NSX upgrade process, the upgrade may fail when attempting to install software components (VIBs) on an underlying ESXi host. The user observes an error indicating an SSL handshake failure caused by an expired certificate on the ESXi host.
  
  
• The following or a similar error message is displayed on the NSX UI:
  
  Error: Failed to install software on host. Failed to retrieve existing vib components from host. : java.rmi.RemoteException: VI SDK invoke exception:javax.net.ssl.SSLHandshakeException: Certificate expired for <certificate subject details>
  
  Failed to install software on host. Host #.#.#.# not reachable. java.rmi.RemoteException: VI SDK invoke exception:javax.net.ssl.SSLHandshakeException: Certificate expired for Host
  
  
• The expiration of host's SSL certificate can be confirmed by using this command from SSH on Host
  openssl x509 -in /etc/vmware/ssl/rui.crt -text -noout | grep -A 2 "Validity"
  
  

Environment

VMware NSX

Cause

This happens because of an expired SSL certificate on the target ESXi host where the NSX VIB installation is failing. During the NSX upgrade, the process attempts to establish a secure connection (SSL handshake) with the ESXi host to deploy or update NSX components (VIBs). If the ESXi host's SSL certificate has already passed its expiration date, the SSL handshake fails. This SSLHandshakeException prevents the successful retrieval of existing VIB components and the installation of new NSX software on the host, leading to the upgrade failure.

Resolution

 

Workaround

1. Log in to the vSphere Client (HTML5 client):

   • Use an administrator account.

   • Navigate to: Hosts and Clusters → your_datacenter → your_cluster → impacted_host.

2. Renew the Certificate:

   • Right-click the affected host.

   • Select Certificates → Renew Certificate.

   • Confirm the action. Renewal typically completes within a few moments.

   Note: Self-signed certificates issued by vCenter VMCA or renewed through this method are generally valid for 5 years.

3. Verify the New Certificate:
   Run the following command on the host to confirm that a new certificate has been issued and check the expiration date:

   openssl x509 -in /etc/vmware/ssl/rui.crt -text -noout | grep -A 2 "Validity"
   

4. Restart Management Agents:
   Restart the management agents to ensure the host begins using the newly generated certificate:

   services.sh restart
   

5. Resume NSX Upgrade:
   Retry the NSX upgrade process. After successful renewal, the NSX VIB installation status for the host should show as Successful in the NSX UI.

Additional Note
If the certificate does not renew after selecting Renew Certificate and no error is displayed in the task, follow the Resolution guidance in this KB to validate the value for vpxd.certmgmt:
ESXi host’s certificate cannot renew or refresh when vpxd.certmgmt.mode is not "vmca" or "custom"

Additional Information

Prerequisites before refreshing/renewing the ESXi SSL certificates from vCenter server vSphere UI:

"ESXi Host Certificate Status" alert for any host in vCenter Server

• The ESXi hosts are connected to the vCenter Server.
• Ensure time synchronization between the vCenter Server system and the ESXi hosts.
• DNS resolution works between the vCenter Server system and the ESXi hosts.
  • The vCenter Server system's MACHINE_SSL_CERT and TRUSTED_ROOT certificates are valid and have not expired. See the knowledge base article at Manually reviewing certificates in VMware Endpoint Certificate Store for vSphere 6.x and 7.x.
• The ESXi hosts are NOT in maintenance mode.