Virtual Service Error "VIP Programming In-Progress" After Controller Upgrade

Products

• VMware Avi Load Balancer

Issue/Introduction

After upgrading the Controller to version 31.2.1 while keeping Service Engine (SE) Groups on version 30.2.x, all Virtual Services (VS) may display the following error status:

"Virtual service error: vip programming in-progress on service engine"

Environment

All environments

Cause

This issue stems from a naming mismatch in the internal communication protocol between the Controller and the SE-Agent.

In version 31.2.1, the UUID for internal shards was renamed:

• Old Version (30.2.x): state_cache_mgr

• New Version (31.2.1): statecache_mgr (underscore removed)

When a version 31.2.1 Controller attempts to sync state with an older (30.2.x) Service Engine, it sends instructions referencing statecache_mgr. Because the older SE-Agent does not recognize this new string, it fails to process the request, leaving the VIP in a permanent "programming in-progress" state.

This specifically impacts Upgrade 2.0 workflows where the Controller is upgraded ahead of the Service Engines.

Workaround

To resolve this error immediately and restore Virtual Service functionality, upgrade all Service Engine Groups to the same version as the Controller (31.2.1). Aligning the SE version with the Controller version ensures that both components use the same shard naming convention, allowing the SE-Agent to correctly process the programming requests.

 

Resolution

This naming discrepancy has been resolved to ensure backward compatibility between the Controller and older SE versions.

Fixed Version: 32.2.1