Layer 7 Load Balancer returns HTTP 502 while accessing the VIP in NSX-T Edge

Products

• VMware NSX

Issue/Introduction

Symptoms:
In the /var/log/syslog file in NSX Edge, you see message similar to:

/var/log/syslog.1:<27>1 2021-01-27T13:28:58.745138+00:00 lab00775 NSX 8231 LB [nsx@6876 comp="nsx-edge" subcomp="nsx-edge-lb.lb" level="ERROR" errorCode="EDG9999999"] [735c34e3-34e2-4dae-852a-9baaec590e56] upstream sent too big header while reading response header from upstream, client: <IP>, server: , request: "GET /auth/code?code=OanOh4rBqp&state=69707f90-4ba2-4903-af23-85f468124150 HTTP/1.1", upstream: "https://<IP>:0/auth/code?code=OanOh4rBqp&state=69707f90-4ba2-4903-af23-85f468124150", host: "<HOSTNAME>"

Note : Most of the Tanzu applications are showing these symptoms after Tanzu upgrade.

Environment

VMware NSX-T

Cause

This issue is caused when the LB was not able to process this response , due to which LB sent HTTP 502 to client.
The  HTTP response header sent by backend server to LB was higher in size, than the default value configured at LB. By default, the value of HTTP response header at LB is 4096 bytes.

Resolution

To resolve this issue, Customize based on the HTTP response header size sent by the application:

• If the LB entity is created using Policy , customer can create a HTTP profile at policy UI with higher response header size .
  Path:--Networking-->Load Balancer-->Profiles-->Add Application profile -->HTTP.
  Apply this HTTP profile to VIP at Virtual servers Page.
• If the LB entity is created at MP(either via UI or NCP) , customer can create a HTTP profile at Policy UI and attach the newly created profile to VIP at MP UI.