HTTP/2 Protocol_Error: curl: (92) Invalid HTTP header field received by virtual service due to Invalid Upgrade Headers

Products

• VMware Avi Load Balancer

Issue/Introduction

• Users or monitoring tools (e.g., 'curl') receives protocol errors when accessing a Virtual Service (VS) configured with HTTP/2.
• Browsers may appear unaffected as they often ignore the invalid headers, but stricter clients can fail.
• Error messages include:
  - `HTTP/2 stream 1 was not closed cleanly: PROTOCOL_ERROR (err 1)`
  - `curl: (92) Invalid HTTP header field was received: frame type: 1, stream: 1, name: [upgrade], value: [h2,h2c]`
  
  

Environment

Avi load balancer

Cause

The issue occurs when the following conditions are met:

1. The Virtual Service is configured for HTTP/2.
2. The Backend Pool may be configured for HTTP/2, but the backend server negotiates HTTP/1.1 during the TLS-ALPN negotiations .
3. The backend server includes an `Upgrade: h2, h2c` header in its HTTP/1.1 response.
4. The Avi Service Engine (SE) transparently forwards this `Upgrade` header to the client over the existing HTTP/2 connection.

As the "Upgrade" header is not valid when a client is already using HTTP/2 ; this can result in a "PROTOCOL_ERROR" message for the user. 

Resolution

To mitigate the issue immediately, configure an HTTP Response Policy on the Virtual Service to remove the "Upgrade" header from backend responses.

Steps to apply the HTTP response policy:

1. Navigate to the Application > Virtual Service (VS) - edit VS configuration.
2. Move to the Policies tab > HTTP policy sets > Add - click on the three dots and click create.
3. Provide a name for the HTTP policy set and naviagte to HTTP response rules ; click on Add.
4. Create a new rule as shown in below screenshot and save the rule:  

  

4. Save the HTTP Policy set.