Apps Crash When Applying Changes to Diego Cells
search cancel

Apps Crash When Applying Changes to Diego Cells

book

Article ID: 438468

calendar_today

Updated On:

Products

VMware Tanzu Application Service

Issue/Introduction

Apps are crashing when the diego cells get updated during an apply changes. They fail with a timeout error during the startup check:

   2026-04-20T14:50:00.44-0400 [HEALTH/0] ERR Timed out after 1m0s (31 attempts) waiting for startup check to succeed: failed to make TCP connection to ###.###.###.###:8080: dial tcp ###.###.###.###:8080: connect: connection refused
   2026-04-20T14:50:00.44-0400 [CELL/0] ERR Failed after 1m7.358s: startup health check never passed.

The application shows a delay after invoking the start command:

   2026-04-20T14:48:54.26-0400 [APP/PROC/WEB/0] OUT Invoking pre-start scripts.
   2026-04-20T14:48:54.26-0400 [APP/PROC/WEB/0] OUT Invoking start command.
   2026-04-20T14:48:55.21-0400 [APP/PROC/WEB/0] OUT JVM Memory Configuration: -Xmx400509K -Xss1M -XX:ReservedCodeCacheSize=240M -XX:MaxDirectMemorySize=10M -XX:MaxMetaspaceSize=136066K
   2026-04-20T14:49:41.39-0400 [APP/PROC/WEB/0] OUT .   ____          _            __ _ _
   2026-04-20T14:49:41.39-0400 [APP/PROC/WEB/0] OUT /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
   2026-04-20T14:49:41.39-0400 [APP/PROC/WEB/0] OUT ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
   2026-04-20T14:49:41.39-0400 [APP/PROC/WEB/0] OUT \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
   2026-04-20T14:49:41.39-0400 [APP/PROC/WEB/0] OUT '  |____| .__|_| |_|_| |_\__, | / / / /
   2026-04-20T14:49:41.39-0400 [APP/PROC/WEB/0] OUT =========|_|==============|___/=/_/_/_/
   2026-04-20T14:49:41.44-0400 [APP/PROC/WEB/0] OUT :: Spring Boot ::               (v2.7.14)

In this example, the application had a 46 second delay after calculating the JVM Memory Configuration (14:48:55 -> 14:49:41).

Eventually they recover on their own and start up.

Environment

Elastic Application Runtime

Cause

This can happen when many apps try to start up at the same time, which happens when diego cells get updated. The delay in the startup command indicates that there is resource contention on the diego cells. This could be low CPU, low memory, slow IO, or some other resource constraint.

This is more common when there is a small number of diego cells

Resolution

There are several different ways to solve this problem:

Powered by Wolken Software