How to set autoscaler rule to monitor external Rabbit MQ queue length

Products

• VMware RabbitMQ
• VMware Tanzu RabbitMQ
• Pivotal RabbitMQ

Issue/Introduction

Autoscaler allows to scale an application using Rabbit MQ length as a metric. 

In order to retrieve the queue length, Autoscaler leverages the Rabbit MQ management plugin. This plugin expose an HTTP API. It allows to collect various statistics and manage the system.

The Tanzu RabbitMQ service broker will store the URL of the management API in the service binding credentials. The Autoscaler service will use the Cloud Controller API to retrieve an application's environment variables and extract the Rabbit MQ management API URL. 

The architecture above allows Autoscaler to monitor any Rabbit MQ server as long the conventions are followed.

Environment

Product Version: 2.6

Resolution

When using the Tanzu RabbitMQ Tile & Broker, RabbitMQ & service instances are configured correctly so that this will just work. If you would like to manually try this out, you can run the following steps.

1. The management plugin must be enabled on your RabbitMQ server. On the Rabbit MQ node, execute the following command:

rabbitmq-plugins enable rabbitmq_management

Reference: https://www.rabbitmq.com/management.html

2. Assign administrator privilege to a user on the Rabbit MQ node - execute the following command:

rabbitmqctl set_user_tags <username> administrator

3. Verify access to the management API:

curl -v 'http://<username>:<password>@<rabbit-mq-FQDN>:15672/api/queues'

4. Define a user provided service in cloud foundry:

cf cups <SERVICE_NAME> \
  -p '{"uri": "amqp://<USERNAME>:<PASSWORD>@<RABBIT_MQ_FQDN>", "http_api_uri": "http://<username>:<password>@<rabbit-mq-FQDN>:15672" }' \
  -t rabbitmq

The 2 requirements for the the autoscaler to use it:

1. The service must have the rabbitmq tag
2. The "http_api_url" property must be set.

5. Bind the service to your application and re-stage or deploy application.

6. Using Apps Manager or the cf autoscaler CLI, create the rule and provide the queue name.
 

cf create-autoscaling-rule <APP_NAME> rabbitmq <MIN> <MAX> -s <QUEUE_NAME>

When the queue depth increases/decreases, Autoscaler should now scale up/down your application accordingly.

Please note that some versions of the Tanzu RabbitMQ Tile will store credentials in CredHub. This breaks the process above, because the HTTP API URL is not set. Please see Application Auto-scaling not working: Event RabbitMQ Message Queue for more details.