"Error: organization's memory limit exceeded: staging requires 1024M memory" error after restaging app fails if available org memory is smaller than app memory limit in Tanzu Application Service for VMs

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

After upgrading to Tanzu Application Service for VMs (TAS for VMs) 2.11+, if the available org memory is smaller than the app memory limit, restaging that app fails with an error similar to following:

"Error: organization's memory limit exceeded: staging requires 1024M memory"

The required amount of memory varies per the app memory limit. The above message will be shown for an app with 1 GB memory limit or an app that has 1 GB memory allocated. 

Environment

Product Version: 2.11

Resolution

There are a few of options to resolve this issue: 

• Update the org quotas so there is enough space for the staging process to run.
• Scale down the app memory in Apps Manager or with the command: cf scale APP-NAME> -m <MEMORY>
• Delete applications that are not being used in that org.

This issue is caused by the bug fix V3 Apps do not get the proper amount of memory for staging #1802, where staging apps could fail due to insufficient memory allotted. This fix was released in Cluster API (CAPI) 1.100.0. Any TAS for VMs version containing this CAPI version or a later version experience this behavior. 

In TAS for VMs 2.11+, the memory required for staging containers is now equal to the memory size of the app. In other words, an org needs to have at least the same amount of free memory as the app memory limit to prevent restaging from failing. 

For all applications to be able to restage, there has to be as much free memory as the largest memory limit that an application has in the space. In other words, If you have 3 apps in a space: app1, app2 and app3 with 1GB, 2GB and 3GB allocated respectively, you would need at least 3 GB free in the org so all apps can restage. If you had 2 GB, only app1 and app2 would be able to restage.

Note: This article uses the following terms indistinctly: memory limit, memory size and memory allocation; meaning the memory reserved for an application when using -m flag in "cf push" command.