Adding Custom Certificate on ESXi hosts through CLI

Products

• VMware vSphere ESXi

Issue/Introduction

The KB outlines the steps to add custom certificate as the root CA to the ESXi trusted domain without bypassing the certificate based SSL authentication. The root CA can then be used to sign other intermediate CERTs and/or the host certificate file (i.e. private key – public key pair).Before making any changes you may like to validate with customer if they are using any third party trusted certificates.

Resolution

Process to add Custom Certificate on ESXi hosts through CLI:

1. Set the vCenter Server to custom certificate mode by following the steps below: 

a) In the vSphere Client, select the vCenter Server that manages the hosts.
b) Click Configure, and under Settings, click Advanced Settings.
c) Click Edit Settings.
d) Click the Filter icon in the Name Column, and in the Filter box, enter vpxd.certmgmt to display only certificate management parameters.
e) Change the value of vpxd.certmgmt.mode to custom if you intend to manage your own certificates and click Save.
f) Restart the vpxd service using below command:

service-control --restart vpxd

2. Ensure the custom Root certificate is retrieved in advance before proceeding.
3. Place the ESXi host in maintenance mode (Evacuate all data to other hosts)
4. Disconnect the ESXi host from the cluster.
5. SSH into the ESXi host
6. Run this command to take a backup of the castore.pem file:  

cp /etc/vmware/ssl/castore.pem /etc/vmware/ssl/castore.pem.bak

Note: The system will be rebooted in the following steps. If user wants to access the castore.pem.bak in future probably, please copy (using scp) this file out from the ESXi host.

7. Copy the Root certificate to /etc/vmware/ssl/Root.cer

Note: If the user has one or more intermediate certificate authorities, the Root certificate must be a chain of all intermediate certificates and root certificate. 

8. Append the Root certificate to castore.pem file by command:

cat Root.cer >> castore.pem

Note: User can append multiple root certs however the ESXi host certificate file should be signed by one root certificate (the pem file should contain, the machine ssl, the intermediate ssl, and the root cert)

9. Delete the Root certificate by command: 

rm Root.cer

10. Replace the default rui.crt and rui.key with trusted CA-signed certificate and key per Replace the default Certificate and Key from the ESXi Shell.
11. After applying the custom certificate in ESXi hosts, the user needs to persist those changes into the system disk by running this command:

/sbin/auto-backup.sh

12. Restart the ESXi host.
13. Reconnect the ESXi host back to the original cluster.
14. Exit maintenance mode. 

NOTE: This process can be used on a vSAN cluster to authenticate hosts utilized in the vSAN cluster. As Chrome and Edge does not give option to download the PEM CERTS , the customer may use Mozilla Firefox for Windows.