Error: "Operation failed" when installing custom SSL certificate in VCF/Aria Operations.

Products

• VCF Operations

Issue/Introduction

• When installing a new certificate in VCF Operations using the steps in the procedure Configure a Certificate For Use With VCF Operations, the following error is reported after selecting the certificate PEM file: 

Operation failed. If the error persists, contact VMware support.

• Cannot upload custom SSL certificate PEM file which is using double authority/multi-level CA chain and getting the same error message.
  
  
• An error similar to the following is seen when running the Validate a PEM file steps in KB: Using the Custom Certificate Tool in VMware Aria Operations.

$VMWARE_PYTHON_BIN vropsCertificateTool.py -i uploaded_cert.pem
Traceback (most recent call last):
  File "/usr/lib/vmware-casa/bin/vropsCertificateTool.py", line 478, in _parse
    self._parsed_object = Certificate(self.pem_data)
  File "/usr/lib/vmware-casa/bin/vropsCertificateTool.py", line 175, in __init__
    self._certificate_data = self.load_certificate(self._pem_data)
  File "/usr/lib/vmware-casa/bin/vropsCertificateTool.py", line 307, in load_certificate
    return OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, pem_data)
  File "/usr/lib/python3.10/site-packages/OpenSSL/crypto.py", line 2034, in load_certificate
    _raise_current_error()
  File "/usr/lib/python3.10/site-packages/OpenSSL/_util.py", line 57, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.crypto.Error: [('PEM routines', '', 'bad end line')]

Environment

• VMware VCF Operations 8.18
• VMware Aria Operations 8.18.x

Cause

This issue is caused by one of the certificates used to build the PEM certificate chain not being imported correctly causing it to not be a valid PEM chain certificate.

Examples of an invalid certificate could be:

1. Missing newlines between certificate elements, for example:
   
   -----BEGIN CERTIFICATE-----
Content of the Signed certificate.
-----END CERTIFICATE----------BEGIN PRIVATE KEY-----
Content of the private key.
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
Content of the Intermediate CA certificate
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Content of the Root CA certificate
-----END CERTIFICATE-----
   
   
2. Certificate using double authority/multi-level CA chain, even if openssl verify passes, the Admin UI parser may still fail if the chain isn't correctly consolidated.

Resolution

For a standard certificate. to apply the following steps:

1. Using a text editor such as (vi) edit the PEM chain cert so that each cert is on its own line, example below:

-----BEGIN CERTIFICATE-----
Content of the Signed certificate.
-----END CERTIFICATE-----
-----BEGIN PRIVATE KEY-----
Content of the private key.
-----END PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
Content of the Intermediate CA certificate
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Content of the Root CA certificate
-----END CERTIFICATE-----

2. Once the end line of the certificate is corrected the certificate can be verified with commands specified in Using the Custom Certificate Tool in VMware Aria Operations.

A valid certificate should have an output similar to:

$VMWARE_PYTHON_BIN /usr/lib/vmware-casa/bin/vropsCertificateTool.py -i uploaded_cert.pem

Found section: CERTIFICATE
    description:
        subject = <subject>
        issuer = <Intermediate CA>
Found section: PRIVATE_KEY
    description:
        Key Size = 2048 bits
Found section: CERTIFICATE
    description:
        subject = <Intermediate CA>
        issuer = <Root CA>
Found section: CERTIFICATE
    description:
        subject = <Root CA>
        issuer = <Root CA>
Input file is valid.

If you are using double authority/multi-level CA chain, you must combine your custom leaf certificate and the entire CA chain (Intermediate and Root) into a single, valid PEM-encoded text file.

Here the steps to follow:

1. Open the PEM file in a text editor (e.g., vi or Notepad++).
2. Ensure each certificate and the private key start on a new line.
3. If using a multi-level or double Certificate Authority (CA), consolidate the leaf certificate, all intermediate CAs, and the root CA into the single PEM file in the following order:
   • Private Key
   • Leaf Certificate
   • Intermediate CA(s)
   • Root CA
4. Verify the file using the Custom Certificate Tool.