Virtual appliance performance optimization and NUMA locality for CA API Gateway

Products

• CA API Gateway

Issue/Introduction

The CA API Gateway virtual appliance is a latency-sensitive (latency-bound) application. Its primary design objective is to process each individual request or packet with the lowest possible latency to ensure fast API response times. While the Gateway scales for high throughput, performance bottlenecks can occur if the underlying virtual machine (VM) configuration does not respect Non-Uniform Memory Access (NUMA) boundaries.

Environment

• CA API Gateway (All Versions)
• VMware ESXi

Resolution

To ensure optimal performance and minimize memory access latency, follow these best practices for VM configuration:

1. Size for NUMA Locality: Configure the VM's vCPU and memory so that the entire allocation fits within a single physical NUMA node on the ESXi host. If a VM spans multiple physical NUMA nodes, the guest OS may experience "remote memory" access, significantly increasing processing latency.

2. Disable Memory Hot-Add: Ensure the Memory Hot-Add feature is disabled in the VM settings. When Memory Hot-Add is enabled, ESXi often hides the vNUMA topology from the guest operating system. Disabling this feature allows the Gateway's Linux-based OS to see and optimize for the underlying vNUMA structure.

3. OS-Level Latency Tuning: For Software Gateway deployments, ensure the following kernel parameter is set to prioritize low latency: net.ipv4.tcp_low_latency = 1

4. Baseline Performance Tuning: Refer to existing documentation for threading and database connection pool optimizations to complement NUMA-level settings.

Additional Information

• Using Virtual NUMA Controls (KB 323386)