Starting or Staging an Application Results in an Insufficient Resources Error in PCF/TAS

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Symptoms:

When starting or staging an application, the cf cli returns the insufficient resources error. The error message looks like the following:

2016-03-16T15:19:58.16+0100 [API/0] ERR Failed to stage application: insufficient resources

Environment

Cause

When getting this error, it means that you do not have enough free, unreserved space in the environment to either start or stage the application. To start an application, you need to have at least the requested memory limit in free memory and the requested disk quota in free disk space. To stage an application, you need to have at least 1G of memory or the requested memory limit free, whichever one is larger, and you need to have at least 6G of free disk space. If you do not meet these requirements, starting or staging the app will fail with an insufficient resources error.

Resolution

The simplest resolution is to add more Diego Cells. This will increase the available capacity on the platform and should clear up the insufficient resource errors.

If you are skeptical that the platform is actually low on resources, you can perform the following steps to investigate more:
 

1. Check the total memory and disk metrics. Make sure this is showing expected values based on the memory and ephemeral disk size of your Diego Cells. Note: Some of the ephemeral disk space will be lost to swap. See this article for more details on Bosh deployed VMs and swap space.
2. Check the available memory and disk metrics. Make sure that at least one Diego Cell has enough memory and disk space to run or stage your application. See the Cause section above for details on how much your application will need to run or stage. Also, confirm that the available container count has not hit zero.
3. Check that your Cells are all reporting as healthy. You can see this by looking at the metric opentsdb.nozzle.rep.UnhealthyCell. When this returns 0, it means the cell is healthy. When it returns 1, it means the cells are unhealthy. Unhealthy cells will not take on new work and that will lower your capacity.

If you need additional assistance investigating why you're receiving insufficient resource errors, please open a support ticket.