Packet Drops with Active/Active or N+M Avi Load Balancer Service Engines in a Cisco VXLAN/ACI Environment
search cancel

Packet Drops with Active/Active or N+M Avi Load Balancer Service Engines in a Cisco VXLAN/ACI Environment

book

Article ID: 406213

calendar_today

Updated On:

Products

VMware Avi Load Balancer

Issue/Introduction

When an Avi load balancer with a Service Engine (SE) Group configured for Active/Active or N+M high availability (with minimum scale-out set to 2 SEs) is deployed in a Cisco VXLAN or ACI environment, intermittent packet loss may occur.

Symptoms can include impact to the virtual service datapath, dropped client connections, and health monitor failures for backend servers and GSLB pool members.

Environment

Cisco ACI

Cisco VXLAN

Cause

  • The issue is caused by a MAC address conflict within the Cisco VXLAN/ACI fabric due to the nature of Avi's Active/Active configuration.
  • In an Active/Active setup, a single Virtual IP (VIP) is used, but client connections are distributed across both the primary and secondary Service Engines.
  • This means that return traffic for the same VIP can be sourced from two different MAC addresses (one for the primary SE and one for the secondary SE).
  • Cisco VXLAN/ACI fabrics often operate with a strict mapping of one IP address to one MAC address.
  • When the fabric sees packets for the same VIP originating from the secondary SE's MAC address after previously seeing them from the primary SE's MAC address, it can interpret this as a MAC flap or security violation and subsequently drop the packets.

Resolution

  • The resolution is to enable Tunnel Mode on the affected Service Engine Group within the Avi Controller.
  • Enabling Tunnel Mode modifies the return traffic flow.
  • All traffic from a secondary SE is first sent through a secure tunnel to the primary SE.
  • The primary SE then sends the traffic to the client.
  • This process ensures that all return traffic for the VIP is consistently sourced from the single MAC address of the primary SE, which resolves the conflict with the Cisco VXLAN/ACI fabric.

Steps to enable tunnel mode on Avi service engine group:

  1. SSH into the Avi controller leader VM.
  2. Run the commands below: 
    • $shell
      > configure serviceenginegroup <se-group-name>
      > se_tunnel_mode 1
      > save
  3. This configuration change is effective immediately and does not require a reboot of the service engines.

For a detailed explanation of the packet flow when Tunnel Mode is enabled, refer to this article: Packet Flow in Avi Load Balancer with Tunnel mode enabled

Additional Information

Powered by Wolken Software