TAS S3 BlobstoreVerifier Error SSL/TLS error

Products

• Operations Manager

Issue/Introduction

Symptoms:

Error Message:

Errors::CertificateError SSL_connect returned=1 errno=0 state=SSLv3 read
server certificate B: certificate verify failed (OpenSSL::SSL::SSLError) Unable to verify certificate 

Environment

Cause

When you press the Save button or when you Apply Changes, Operations (Ops) Manager attempts to validate the settings entered to ensure that they are correct. To validate the File Storage settings of the Pivotal Application Service tile, Ops Manager will attempt to connect to the specified S3 compatible Blobstore. Because the Blobstore is not presenting a trusted certificate, Ops Manager is unable to make the connection and fails with the above error.

 

Resolution

Firstly, confirm if your S3-compatible Blobstore is configured with a self-signed certificate. When it is configured so, you need to add a custom certificate to the Ops Manager Virtual Machine's (VM's) trust store. After that, Ops Manager will be able to successfully validate the external S3 Blobstore configuration. 

Here are the steps to accomplish this:

1. SSH to Ops Manager VM. Run`ssh ubuntu@opsmanagerFQDN`.

2. Ru `sudo -i`. You'll be prompted again for the `ubuntu` user's password.

3. Change directories with `cd /usr/local/share/ca-certificates/`

4. Create a file in the current directory with the name s3-custom.crt and paste in the contents of the custom certificate (including the complete BEGIN & END CERTIFICATE lines).
   
   Ex:

   cat <<EOF > s3-custom.crt
   <cursor will sit here -> paste in your cert & press ctrl+d to end input>
   

5. Run `update-ca-certificates`. This will update the VM's list of trusted certificates.

6. You should see a message that certificate was added, like this.

   root@bosh-stemcell:/usr/local/share/ca-certificates# update-ca-certificates
   Updating certificates in /etc/ssl/certs... 1 added, 0 removed; done.
   Running hooks in /etc/ca-certificates/update.d....
   Adding debian:cert.pem
   done.
   done.
   

7. Repeat this attempt to save the configuration or click, "Apply Changes". Ops Manager should now be able to validate your custom certificate.

Finally, in Ops Manager navigate to Security from Director. In the BOSH Trusted Certs box, add the contents of the self-signed certificate that is being used by your S3 compatible Blobstore. Click Save. When you apply changes, this will ensure that the certificate is pushed out to all of the VMs deployed by BOSH ensuring that they will be able to communicate with your S3 compatible Blobstore.