Streaming Error (422) in AI Services on Tanzu Platform 10.4.3 When Using vLLM Endpoints

Products

• VMware Tanzu Platform Core

Issue/Introduction

When running streaming chat requests through the AI Services proxy on Tanzu Platform 10.4.3 against an off-platform, vLLM-based OpenAI-compatible endpoint, the request fails with a 422 Unprocessable Entity error.

The backend rejects the request due to extra inputs being added by the proxy, specifically:

422 VALIDATION_ERROR loc:["body","stream_options","include_obfuscation"] "Extra inputs are not permitted" (extra_forbidden)

Environment

Tanzu Platform 10.4.3 (Air-gapped)

Cause

The AI Services proxy injects stream_options.include_obfuscation into streaming requests by default. When the backend vLLM-based service is configured to forbid extra inputs (extra=forbid), this unexpected field causes a validation error.

Resolution

This has been identified as a bug within the AI-Server. Engineering is tracking and working on a fix for release. This KB will be updated once the fix is released.