Avi Service Engine Packet Buffer Leak After Upgrading to Version 30.2.x

Products

• VMware Avi Load Balancer

Issue/Introduction

After upgrading Avi to a 30.2.x release (<30.2.7), packet buffer usage may gradually build up over a few days, indicating a packet buffer leak.

Symptoms:

• This depletion of packet buffers can negatively impact traffic flows.
• The 'Buffer usage' graph under the Service Engine 'Analytics' tab increases gradually over time.
  • 
• The following events may also be observed in the service engine 'Events' page:
  • SE_PKT_BUFF_HIGH
  • CONN_DROP_NO_PKT_BUFF
  • PKT_DROP_NO_PKT_BUFF
  • 

Environment

Avi versions 30.2.1 to 30.2.6

Cause

In IPv6-based environments, Path MTU (PMTU) discovery behavior can trigger a packet buffer leak on the Service Engine, leading to the gradual exhaustion of available buffers.

Resolution

Permanent fix:

This issue is permanently resolved in Avi version 30.2.7. Once the version is available, upgrading to this release is recommended.

Workarounds:

You can apply one of the following mitigations:

• Option 1: Disable IPv6 PMTU Discovery (Recommended):
  • You can disable PMTU discovery for IPv6 via the Service Engine Group configuration. Execute the following commands in the Avi CLI:

  • > configure serviceenginegroup <se-group>
    > no path_mtu_discovery_v6
    > save

• Option 2: Reboot the Service Engines:
  • Rebooting the affected Service Engines will clear the depleted buffers and temporarily restore standard operations.
  • However, buffer usage will eventually climb again over time until the configuration is changed or the system is upgraded.

 

Additional Information

Support for Jumbo Frames in Avi Load Balancer: https://techdocs.broadcom.com/us/en/vmware-security-load-balancing/avi-load-balancer/avi-load-balancer/30-2/se-advanced-networking/support-for-jumbo-frames-in-avi-vantage.html