Best practices for handling sudden spikes in Ingress in Tanzu Elastic Application Runtime

Products

• VMware Tanzu Application Service
• VMware Tanzu Platform - Cloud Foundry

Issue/Introduction

Operator is looking for best practices for handling sudden spikes in Ingress in Tanzu Elastic Application Runtime.

Applications crash during sudden ingress spikes before autoscaler can increase the number of instances.

How to prevent application overruns and crashes during sudden increases in traffic.

Environment

This KB applies to products: 

• Tanzu Elastic Application Runtime
• Tanzu Application Service
• Tanzu Platform Cloud Foundry

Resolution

The following strategies can be used to prevent application crashes and overruns during sudden spikes in ingress in Tanzu Elastic Application Runtime

 

Proper Health Checks for Container Resiliency

• Readiness Probes: Improve resiliency of your readiness endpoint (e.g., Spring Boot Actuator /actuator/health) to include downstream dependencies (DB, Kafka, etc.).
• This tells the Cloud Foundry Gorouter to stop sending ingress traffic to instance nodes that are already overloaded or degraded
• Frequent readiness healthchecks to remove the app from the routing pool 

Autoscaling Tuning

• Set lower autoscaling thresholds to more aggressively scale application instances during high ingress spikes.

Front-load Application Instances and Resources

• Starting with more app instances already running to avoid any overruns.

Optimize Tomcat Threading and Connection Pooling

• Application developer can consider limiting max-threads in Spring Boot applications, 
• Cap your Tomcat threads in application.yml or application.properties so the app rejects or queues requests rather than crashing.
• Unbounded request threads during spikes cause massive memory allocation and database bottlenecks.