Creating custom certificates for vCenter Server with OpenSSL and Microsoft Certificate Services Web Enrollment

Products

• VMware vCenter Server

Issue/Introduction

This article provides instructions to create custom SSL certificates for VirtualCenter/vCenter Server with OpenSSL and Microsoft Certificate Services Web Enrollment.

Note: This article assumes that you have already installed and configured a Microsoft Certificate Services Web Enrollment Root Certificate Authority (CA).

Environment

VMware VirtualCenter 2.5.x
VMware vCenter Server 4.1.x
VMware vCenter Server 4.0.x
VMware vCenter Server 5.0.x

Resolution

To create custom certificates:

Note: Run all commands from c:\openssl\bin.

1. Download and install OpenSSL for Windows.
2. Open the c:\openssl\bin\openssl.cnf file and enter your default information.
   
   Note : This step saves time later in the procedure.
   
   
3. Follow the steps for your system:
   
   
   • For 32-bit systems:
     
     
     1. Create the vCenter Server private key rui.key with this command:
        
        openssl genrsa 1024 -out rui.key


     2. Create the certificate signing request rui.csr with this command:
        
        openssl req -new -key rui.key -out rui.csr -config openssl.cnf

        Note: When prompted for the common name, enter the hostname of the vCenter Server.
        
        
   • For 64-bit systems:
     
     
     1. Run the openssl command prompt from the installation directory C:\Users\Administrator\Desktop\openssl-0.9.8k_X64\bin\openssl.exe if installed in C:\Users\Administrator\Desktop\.
        
        
     2. Run this command:
        
        genrsa -out rui.key 1024


     3. Create the certificate. When creating the certificate file, use the full path to the rui.key and openssl.cnf files:
        
        openssl req -new -key "C:\Users\Administrator\Desktop\openssl-0.9.8k_X64\bin\rui.key" -out rui.csr -config "C:\Users\Administrator\Desktop\openssl-0.9.8k_X64\openssl.cnf"
        
        
4. Obtain the signed certificate rui.crt:
   
   
   1. Open Internet Explorer, and browse to the Microsoft Certificate Services Web Enrollment server (Root CA). The URL is similar to:
      
      http://root_ca/certsrv
      
      
   2. Click Request a certificate > advanced certificate request.
   3. Click Submit a certificate request by using a base-64-encoded CMC or PKCS #10 file, or submit a renewal request by using a base-64-encoded PKCS #7 file.
   4. Open c:\openssl\bin\rui.csr in a text editor. Select and copy the contents of the file.
   5. Paste the contents of the rui.csr file into the Saved request field.
   6. Click Submit.
   7. Have your certificate administrator issue your certificate on the MS Root CA.
   8. Return to the home of the Web Enrollment server. Your request, along with the time you submitted it, is listed.
   9. Click on your request.
   10. Select Base 64 encoded.
   11. Click Download CA certificate.
   12. In the File Download window, browse to c:\openssl\bin and save the file as rui.crt.
       
       
5. To merge the server private key and the signed certificate into the file rui.pfx, run this command:
   
   openssl pkcs12 -export -in rui.crt -inkey rui.key -name rui -passout pass:testpassword -out rui.pfx
   
   Note: The password, testpassword, must not be changed.
   
   
6. To replace the certificates on vCenter Server:
   
   
   1. Copy the existing rui.key, rui.crt, and rui.pfx files from C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\SSL\ to a backup folder.
      
      
   2. Copy the custom rui.key, rui.crt, and rui.pfx files to C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\SSL\.
      
      
7. To load the new Certificates:
   
   
   • For vCenter Server 4.1 and later:
     
     
     1. Use the Managed Object Browser (MOB) to reload the SSL Certificate. For more information, see Replacing vCenter Server 4.1 and 5.0 SSL certificates using the vpxd -p command fails with the error: failed to do early initialization (1030661).
     2. Restart the VirtualCenter Management Webservices service. For more information, see Stopping, starting, or restarting vCenter services (1003895).
        
        
   • For VirtualCenter and vCenter Server 4.0 and earlier:
     
     
     1. Stop the VirtualCenter Server service. For more information, see Stopping, starting, or restarting vCenter services (1003895) .
     2. To reset your database password, browse to the root directory for your vCenter Server install and run this command:
        
        vpxd.exe –p


     3. When prompted for your new password, enter your existing database password.
     4. When prompted to confirm your password, re-enter the password.
     5. Start the VirtualCenter Server service. For more information, see Stopping, starting, or restarting vCenter services (1003895).
     6. Install the root CA certificate into the trusted root CAs on the vCenter Server.
     7. Log into your vCenter Server using your new certificate.
     8. Reconnect all of your hosts. This forces them to communicate using the new SSL certificate.