Operations Manager S3 Blobstore Configuration Error, "Unable to verify certificate"

Products

• Operations Manager

Issue/Introduction

Symptoms:

When attempting to add S3 Compatible Blobstore to Director Config on Director Tile in Operations Manager, apply changes fail with "Unable to verify the certificate."

Error Message:

Errors::CertificateError SSL_connect returned=1 errno=0 state=SSLv3 read
server certificate B: certificate verify failed (OpenSSL::SSL::SSLError) Unable to verify certificate

 

Environment

Cause

There are two issues that you will encounter when trying to use an S3 compatible blobstore that has a custom cert.  The first is that Operations Manager will not be able to successfully validate the connection to your blobstore.  This presents as the error message listed in the symptoms section above.  This can be resolved by manually trusting the custom certificate or CA on the Operations Manager VM.  However, you cannot resolve the second issue which is that the custom certificate or CA will not be trusted on BOSH Director.  In this case, when you click Apply Changes the installation will start and fail halfway through complaining that it cannot upload items to the blobstore because the certificate is not trusted.

Custom certs for an S3 compatible blobstore are not currently supported on the BOSH Director.

 

Resolution

Firstly, confirm if you are using a self-signed or a public CA certificate when configuring S3.

If using Self-Signed CA

There is no way to install a custom CA cert on the Bosh director up until the 1.12 version. For now, the options are switch to use the internal blobstore with the Bosh Director or install a trusted certificate on your S3 compatible blobstore.

If using a Public CA Signed certificate then it should just work.  If you are seeing an issue, you can use the following instructions to confirm that your certificate is trusted.

1. Only server certificates signed by a publicly signed Certificate Authority whose public cert exist in the /etc/ssl/certs/ on the Ubuntu stemcell will work.

2. Check this Directory for the listings of the server certificates of known Certificate Authorities.  For the certificate to be trusted, your CA will need to be listed here.

ubuntu@<opsmanager>:# ls /etc/ssl/certs 
...
Verisign_Class_3_Public_Primary_Certification_Authority_-_G3.pem
VeriSign_Class_3_Public_Primary_Certification_Authority_-_G4.pem
VeriSign_Class_3_Public_Primary_Certification_Authority_-_G5.pem
Verisign_Class_3_Public_Primary_Certification_Authority.pem
VeriSign_Universal_Root_Certification_Authority.pem
Visa_eCommerce_Root.pem
WellsSecure_Public_Root_Certificate_Authority.pem
WoSign_China.pem
WoSign.pem
XRamp_Global_CA_Root.pem

3. Verify that the certificate presented by the S3 compatible blobstore configured under the Operations Manager Director Tile is signed by one of the well known Certificate Authorities listedin the /etc/ssl/certsdirectory.