How to delete orphaned service instances derived from service brokers in Operations Manager

Products

• Operations Manager

Issue/Introduction

This article explains how to delete those orphaned service instances in Operations Manager.

Suppose that you delete service broker tiles (such as MySQL, Redis, etc.) before unbinding and deleting all the service instances derived from them.

In this case, you may not be able to delete those service instances with just the "cf unbind-service" and the "cf delete-service" commands because it failed to unbind these service instances originally. 

Server error, status code: 502, error code: 10001, message: An unbind operation for the service binding between XXXXX and service instance YYYYY failed: The service broker could not be reached: http://192.168.100.163:8080/v2/service_instances/..../service_bindings/.....?accepts_incomplete=true&plan_id=.....&service_id=.....

This behavior happens in the following scenario:

1. Installing a certain service broker tile, such as MySQL.

2. Installing another service tile that requires service instances derived from the existing service broker installed at step 1 and then the service tile creates service instances automatically.

3. Deleting the service broker tile installed in step 1 without noticing the service tile has created service instances in step 2. Now those service instances get orphaned from the service broker.

Resolution

The basic idea is to use "cf purge-service-instance" command rather than "cf unbind-service" and "cf delete-service" commands as shown below:

$ cf login -a ....  
$ cf target -o someorg -s somespace # Target org and space for orphaned service instances
$ cf service <service instance name> --guid ## Save this GUID. You will use it to identify the corresponding BOSH deployment in the final steps.
$ cf purge-service-instance <service instance name>

After this, please confirm whether the BOSH deployments for the deleted target service instances VMs are actually deleted with "bosh vms" command, etc.

Note: Regarding the deployment name, it may contain the service instance GUID, which is extracted at the previous step before executing "cf purge-service-instance".

You will still see "orphaned" service instance VMs - delete them manually as shown below:

$ ssh ubuntu@... # login to Ops Manager VM or jumpbox via ssh
$ bosh -e <env name>  vms # confirm whether "orphaned" VMs still exist
$ bosh -e <env name> -d service-instance_<SERVICE GUID> delete-deployment [--force] # delete oprhaned VMs if they still exist