AutoScaler Fails to Scale Due to App Instance Quota has been reached

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

In this example, there are applications where the autoscaler rules are set. The rules are set as follows:

• Min instances 4, Max instances 8
• There also is a rule for CPU, min is 20%, max is 80%

Now there are 4 instances of this application running, and they all are over 90%. However, autoscaler is not working with this app.

As a troubleshooting step you enabled debug and verbose logging on the autocaler app:

1. cf t -o system -s autoscaling
2. cf set-env autoscale LOG_LEVEL debug
3. cf set-env autoscale LOG_VERBOSE true
4. restage application

Within the autoscaler debug logs you find the following line corresponding to the affected application:

2019-10-18T12:40:58.19-0400 [APP/PROC/WEB/0] OUT time="2019-10-18T16:40:58Z" level=info msg="Unable to scale. App instance quota has been reached. for app <guid> in space <guid>"

Environment

Product Version: 2.3

Resolution

The message "Unable to scale. App instance quota has been reached." actually means that any of the quotas, including memory, could have been exceeded. It doesn't necessarily mean that there's an app instance quota.

You will want to check the Spaces and Orgs for any quota that's been exceeded and adjust the quotas accordingly.

For memory quota, the following cf plugin works great: https://plugins.cloudfoundry.org/#report-memory-usage