Very high total CPU usage of all load balancers Alarm

Products

• VMware NSX

Issue/Introduction

Title: Alarm for very high total CPU usage of all load balancers.

Event ID: load_balancer. total_lb_cpu_usage_is_high

 

Alarm Description:

• Purpose: To inform users that the total CPU usage of all load balancers is high.
• Impact: This will impact overall service quality, potentially leading to high latency and connection timeouts for clients.

Environment

VMware NSX

Cause

This issue is typically caused by too many resource-intensive load balancers being aggregated on the Edge.

Resolution

Steps to resolve
For 9.0.0 and higher

Recommendation Action:

1. To optimize traffic distribution across Edge nodes, we suggest rebalancing heavy load balancers by relocating them to alternative Edge nodes. To determine the optimal load balancer relocation strategy, provide the Edge ID to invoke the Load Balancer Scale Runbook API at /infra/sha/runbook-invocations/<invoke_id> with a unique <invoke_id>, e.g. lb_scale_4b17dca6. This API will provide detailed action plans for relocating the load balancer.
2. Alternatively, you can manually find a suitable Edge by checking the current capacity on other Edge nodes in the Edge cluster. Refer to the Configuration Maximums documentation to understand the Edge load balancer capacity limits and identify Edge nodes that can accommodate the load balancers.
3. You can manually relocate the load balancers by invoking PATCH /policy/api/v1/infra/tier-1s/<tier-1-id>/locale-services/<locale-services-id> with the identified Edge node ID in the request body.
4. If you fail to identify a suitable Edge node, deploy a new Edge node and migrate the load balancer services from the current Edge nodes to the newly deployed node.

Maintenance window required for remediation? Yes