Migrate Redis OnDemand Service Instances to new AZ

Products

Redis for VMware Tanzu

Issue/Introduction

This article describes the procedure to migrate Redis On-Demand Services instances from one AZ to another.

Environment

Resolution

Summary of procedure:

Create new Redis Service Plan, selecting new AZ
Create new Redis Service Instance
Migrate data to new service instance, select 1 or 2:

Path 1: No data migration, Lower Downtime
Path 2: No data loss, higher Downtime

Remove old Service Instances

Create Redis Service Plan and Service instance

Enable Advanced mode in OPS Man as per KB. This will allow you to modify AZ, subnets etc.
Logon to OPS Man and make sure the subnets used by Redis Tile span the new AZ.

Select Bosh Director Tile
Create Networks
Select Networks used by Redis Tile
Add new AZ to Availability Zones (Do not remove any existing AZs)

Create a new Redis Service Plan on OPS Man. Be sure to configure the new on-demand plan to match the configuration of your existing on-demand plan. The only exception is the AZ Placement, you must select the new AZ only.
Disable Advanced Mode as per the KB above.
Run Apply Changes, selecting Bosh Director and Redis Tile. Make sure to enable errand to Register On Demand Broker
Verify that the service is available in the marketplace and the relevant orgs have access to it. Enable Service access if required
```
cf marketplace
cf service-access
cf enable-service-access p.redis -p PLAN
```
Create a new service using the new service plan
```
cf create-service p.edis PLAN SERVICE_INSTANCE
```
Verify that it has been created in new AZ
```
bosh vms --column=Instance --column=AZ
```
Proceed to the next phase to migrate the data to the new instance.

Migrate Data to new Service Instance

Select path 1 or 2 below depending on whether data migration or downtime is the priority.

Path 1: No data migration, Lower Downtime

If the data present is not required to persist, the migration strategy is as simple as unbinding an application from an On-Demand service instance and rebinding to another On-Demand service instance.

Unbind the application from the On-Demand service instance.
```
cf unbind-service APPLICATION SERVICE_INSTANCE
```

This stops all traffic to the Redis instance

Bind your application to the new On-Demand Service instance
```
cf bind-service APPLICATION SERVICE_INSTANCE
```
Verify that the application is able to communicate with the new service instance

Path 2: No data loss, Higher Downtime

There are cases where Redis data will need to be transferred to the new On-Demand service instance during a migration. This process involves a higher downtime and additional steps.

Backup instance

Unbind the application from the On-Demand service instance.
```
cf unbind-service APPLICATION SERVICE_INSTANCE
```

This stops all traffic to the Redis instance in order to start backups

Identify On-Demand Service Instance to be backed up

Identify the service instance GUID by running
```
cf service SERVICE_INSTANCE --guid
```
Using the GUID you identified above, SSH onto your service instance by running
```
bosh -d service-instance_GUID ssh
```

Take a manual backup. You can create a snapshot which creates an RDB file locally or you can create a snapshot and send it to the remote backup destination.

To create a snapshot only, run:

sudo /var/vcap/jobs/redis-backups/bin/backup --snapshot

This creates a new RDB file in /var/vcap/store/redis-backup

To create a snapshot and send it to the pre-configured remote backup destination, run:
```
sudo /var/vcap/jobs/service-backup/bin/manual-backup
```

Restore instance

Prerequisites for Restore

The instance you are restoring to must have the same resource configuration as the instance you backed up.
Ensure that the persistent disk is large enough to handle the temporary files used during the restore process. It should be 1.5x the amount of RAM in the VM.

Restore Procedure

Copy backup of old Redis instance to Ops Man VM


bosh -d service-instance_GUID scp redis-instance:/BACKUP_LOCATION/BACKUP_FILE.rdb /tmp

Copy backup to new Redis instance from Ops Man VM


bosh -d service-instance_GUID scp /tmp/BACKUP_FILE.rdb redis-instance:/tmp

Using the GUID of the new service instance, SSH onto your service instance by running bosh -d service-instance_GUID ssh
Create a directory under /var/vcap/store, for example my-backups
Copy the RDB file from the /tmp directory to the directory you created above
To verify that the RDB file has not been corrupted run:
```
md5sum PATH-TO-RDB-FILE
```
Compare the value returned in the above step against the contents of the .md5 file that has a similar name to the backup file. The values should be the same.

Restore instance


sudo /var/vcap/jobs/redis-backups/bin/restore --sourceRDB PATH-TO-RDB-FILE

Bind your application to an On-Demand service instance by running:
```
cf bind-service APPLICATION SERVICE_INSTANCE
```

Verify the application is able to communicate effectively with the new service instance

Remove old Service Instances

Once you have migrated to the new Redis instances using Path1 or Path 2 above you can cleanup the old Redis instances

Retrieve the old service instance names and verify that no apps are bound to the service.
```
cf services
```
Delete the service
```
cf delete-service SERVICE_INSTANCE
```