Cannot disable TLS in MySQL for Pivotal Platform v2.7.1

Products

• VMware Tanzu SQL

Issue/Introduction

When upgrading

MySQL for VMware Tanzu from v2.7.0 to v2.7.1 with TLS disabled the upgrade will fail at the smoke test stage. The smoke test fails as the broker is unable to provision a new service instance:

[91mExpected success, but got an error:     <*errors.withStack | 0xc000366180>: {    error: {   cause: {    msg: "cf exited non-zero: Service broker error: contact your operator, service configuration issue occurred\n",   stack: [0x8576b1, 0x857944, 0x856a72, 0x86361c, 0x74a6ec, 0x74a35f, 0x7497f4, 0x7507c6, 0x750074, 0x755fbf, 0x755ad4, 0x755317, 0x75797e, 0x75a617, 0x75a3d9, 0x861994, 0x4f5360, 0x45e791],      },   msg: "create-service failed (service: p.mysql plan: db-1tb name: MYSQL-1-DED-4ecb61b9ba003d7b)",                       },    stack: [0x857a7c, 0x856a72, 0x86361c, 0x74a6ec, 0x74a35f, 0x7497f4, 0x7507c6, 0x750074, 0x755fbf, 0x755ad4, 0x755317, 0x75797e, 0x75a617, 0x75a3d9, 0x861994, 0x4f5360, 0x45e791],    }      create-service failed (service: p.mysql plan: db-1tb name: MYSQL-1-DED-4ecb61b9ba003d7b): cf exited non-zero: Service broker error: contact your operator, service configuration issue occurred  

When trying to create a service instance in broker VM the following error is reported (/var/vcap/sys/log/broker/broker.stdout.log):

[on-demand-service-broker] [b9cc5d04-c0b7-47b0-85c5-3f3393a6b312] 2019/10/14 07:23:07.269672 getting tasks for deployment service-instance_89a71c6a-b482-4485-97eb-a3984746012c from bosh [on-demand-service-broker] [b9cc5d04-c0b7-47b0-85c5-3f3393a6b312] 2019/10/14 07:23:07.322264 service adapter will generate manifest for deployment service-instance_89a71c6a-b482-4485-97eb-a3984746012c [on-demand-service-broker] [b9cc5d04-c0b7-47b0-85c5-3f3393a6b312] 2019/10/14 07:23:07.329717 external service adapter exited with 1 at /var/vcap/packages/odb-service-adapter/bin/service-adapter: stdout: 'contact your operator, service configuration issue occurred', stderr: '[odb-sdk] handling generate-manifest [dedicated-mysql-adapter] 2019/10/14 07:23:07 Failed to update manifest: setting trusted_certificates requires also setting serivce_tls CA [odb-sdk] contact your operator, service configuration issue occurred ' [on-demand-service-broker] [b9cc5d04-c0b7-47b0-85c5-3f3393a6b312] 2019/10/14 07:23:07.329748 generate manifest: contact your operator, service configuration issue occurred [on-demand-service-broker] [b9cc5d04-c0b7-47b0-85c5-3f3393a6b312] 2019/10/14 07:23:07.329755 contact your operator, service configuration issue occurred {"timestamp":"1571037787.329764366","source":"on-demand-service-broker","message":"on-demand-service-broker.provision.unknown-error","log_level":2,"data":{"correlation-id":"405585e5-0953-4e83-566d-85a86c239e99::b13dc49d-5075-4acf-a98d-db69634f0296","error":"contact your operator, service configuration issue occurred","instance-details":{"service_id":"548966e5-e333-4d65-8773-7b4e3bb6ca97","plan_id":"07cf3296-b8e3-4c6b-8d74-b11a89599627","organization_guid":"3376a32e-fa75-442d-9398-e0d41abad7d9","space_guid":"c8eb780a-1113-4c82-9702-bf6d649c74b7","context":{"platform":"cloudfoundry","organization_guid":"3376a32e-fa75-442d-9398-e0d41abad7d9","space_guid":"c8eb780a-1113-4c82-9702-bf6d649c74b7","organization_name":"PENG","space_name":"a9s-test","instance_name":"mysql"}},"instance-id":"89a71c6a-b482-4485-97eb-a3984746012c","session":"16"}} [on-demand-service-broker] 2019/10/14 07:23:07.329842 Request PUT /v2/service_instances/89a71c6a-b482-4485-97eb-a3984746012c Completed 500 in 447.878181ms | Start Time: 2019/10/14 07:23:06.881946 [on-demand-service-broker] [39bd5e52-566c-4bfb-bf1e-787aaef1ade9] 2019/10/14 07:23:07.890425 getting manifest from bosh for deployment service-instance_89a71c6a-b482-4485-97eb-a3984746012c [on-demand-service-broker] [39bd5e52-566c-4bfb-bf1e-787aaef1ade9] 2019/10/14 07:23:07.946052 deleting configs for service-instance_89a71c6a-b482-4485-97eb-a3984746012c [on-demand-service-broker] [39bd5e52-566c-4bfb-bf1e-787aaef1ade9] 2019/10/14 07:23:08.057972 error: error deprovisioning: instance 89a71c6a-b482-4485-97eb-a3984746012c, not found. error for user: instance does not exist. {"timestamp":"1571037788.058011532","source":"on-demand-service-broker","message":"on-demand-service-broker.deprovision.instance-missing","log_level":2,"data":{"correlation-id":"405585e5-0953-4e83-566d-85a86c239e99::b13dc49d-5075-4acf-a98d-db69634f0296","error":"instance does not exist","instance-id":"89a71c6a-b482-4485-97eb-a3984746012c","session":"18"}} [on-demand-service-broker] 2019/10/14 07:23:08.058132 Request DELETE /v2/service_instances/89a71c6a-b482-4485-97eb-a3984746012c Completed 410 in 193.954979ms | Start Time: 2019/10/14 07:23:07.864159

Environment

Product Version: 2.7

Resolution

To fix the problem you need to enable TLS on MySQL for VMware Tanzu v2.7.1 per the below procedure or upgrade to v2.7.2 (supports TLS disabled):

1. Go to Mysql tile -> Security -> Select Optional - Developers may configure their service VMs to use TLS checkbox., Save
2. Go to OpsMan tile -> Security -> Select Include OpsManager Root CA in Trusted Certs checkbox., Save
3. Follow the steps in this document to generate certificate and add it to BOSH Trusted Certificates https://docs.pivotal.io/p-mysql/2-7/prepare-tls.html
4. Go to Review pending changes, select PAS tile and MySQL tile, run Apply Changes for those two tiles.

This has been confirmed as a bug in MySQL for VMware Tanzu v2.7.1 where the option "No TLS configuration" does not work. Therefore in v2.7.1 TLS has to be configured. The option to disable TLS has been restored in v2.7.2.