Why DNS Requests Are Delayed When Using UDP Per-Packet Virtual Service with a Proxy Server

Products

• VMware Avi Load Balancer

Issue/Introduction

• Customers may observe intermittent DNS resolution delays (approximately 5 seconds) when using a proxy server between the client and an AVI Virtual Service (VS) configured with a UDP per-packet Network profile.
• Traffic flow: Client → Proxy Server → Virtual Service → Backend Server
• When a client attempts to resolve a fully qualified domain name (FQDN) through a proxy server, the proxy typically initiates two DNS queries at the same time: one for A record (IPv4) and another one for AAAA record (IPv6).
• Both DNS queries reach the AVI Service Engine (SE) and are forwarded to the backend server. The backend server responds correctly to both queries. However, the SE forwards only the first response (either A or AAAA) to the proxy, causing a timeout on the proxy server.
• After the default timeout of 5 seconds on the proxy side, It retries the queries sequentially (A first, then AAAA). In this case, both queries succeed, resulting in an overall delay of approximately 5 seconds.
• This issue occurs specifically when the Service Engine (SE) receives A and AAAA queries simultaneously while using the UDP per-packet profile.

Environment

• VMware Avi Load Balancer

Cause

• This behavior is due to how the UDP per-packet profile is designed.
• UDP per-packet allows only one outstanding request per flow at a time. Once a response is received from the backend server, the connection is immediately closed.
• This design differs from UDP fast path, which allows multiple outstanding requests.
• When both A and AAAA queries arrive at the same time:
  • Only one request is handled correctly.
  • The second request (typically AAAA) is not properly processed, leading to a delay or timeout.
  • Additionally, proxy servers behave similarly to curl by issuing parallel DNS queries, which makes UDP per-packet unsuitable for this type of deployment.

Resolution

• Use a UDP Fast Path profile instead of UDP per-packet in such environments.
• If a legacy firewall exists in the upstream network, UDP Fast Path may introduce compatibility issues. Refer to the article for more information
• If both UDP per-packet and UDP Fast Path profiles present challenges in your environment, configure a Layer 2 Direct Server Return (L2-DSR) setup.
• This approach helps to avoid the limitations associated with both profiles in such environments. Refer to the configuration guide for more information on L2 DSR.

Additional Information

• UDP per-packet is not suitable when multiple DNS queries (A and AAAA) are sent simultaneously.
• Proxy-based deployments amplify this behavior.
• Switching to UDP Fast Path or implementing L2-DSR can resolve the issue depending on the network environment