How to scale ephemeral disk of On Demand RMQ Service Instances

Products

• VMware RabbitMQ

Issue/Introduction

You are trying to increase the ephemeral disk of the RabbitMQ nodes on the On Demand cluster. For this article, we are trying to increase ephemeral disk from 50GB to 200GB

You can define ephemeral disk size using vm extensions. Let us say we have the following vm_extension name

ephemeral_disk_50GB
ephemeral_disk_200GB 

So you go to RabbitMQ Tile "Global Settings for On-Demand Plans". Scroll down to vm extensions. Removed the old vm extension ephemeral_disk_50GB then added a new vm extension ephemeral_disk_200GB. You then "apply-change"  with the upgrade-all-instances errands turned on but when you look at the existing RabbitMQ On demand Service Instances the RabbitMQ server nodes still has the old size 50GB.

However newly created RabbitMQ service instance will reflect 200GB ephemeral disk.

Environment

Product Version: 2.1

Resolution

This article discuss on how you can change ephemeral disk on existing RabbitMQ On Demand service instances:

1. get the service guid of the service instance 

   cf service <rabbitmq-ondemand-service> --guid

2. get the service instance manifest using the guid from step 1. replace XXX with guid

   bosh -d <service-instance_XXX> manifest > service_instance_XXX.yaml
   

3. make a copy of the manifest. This is just a back up copy in case you made a mistake in editing the yaml file you can just reference the original manifest 

   cp service_instance_XXX.yaml service_instance_XXX_backup.yaml

4. edit service_instance_XXX.yaml. Locate the part vm_extensions

   vm_extensions:
     - ephemeral_disk_200GB
     - ephemeral_disk_50GB

   You can see that both the old vm extension and the newly added vm extensions are present. The ephemeral_disk_50GB will take precedence over ephemeral_disk_200GB.  This would explain why you size is still remained unchanged. Looking into Rabbitmq on demand adapter the order of vm extensions are as follows:  the first on the list of be the VM extensions as entered in the RabbitMQ Tile followed by the VM extensions from the previous manifest
5. Remove the ephemeral_disk_50GB from the manifest so it will look something like 

   vm_extensions:
     - ephemeral_disk_200GB

6. deploy the changes on the manifest

   bosh -d <service-instance-XXXX> deploy /path/service_instance_XXX.yaml

7. Once changes have successfully applied. Check the ephemeral disk size on the vms 

   $ bosh -d <service-instance_XXX> ssh rabbitmq_server_node
   # fdisk -l

8. Repeat steps 1-7 for the rest of the RabbitMQ on demand service instances