Sentinel StatefulSet does not automatically update after ValkeySentinel version change in Tanzu Valkey

Products

• VMware Tanzu Data Suite
• VMware Tanzu for Valkey
• VMware Tanzu Data Suite
• VMware Tanzu Greenplum
• VMware Tanzu Greenplum / Gemfire

Issue/Introduction

After upgrading the Tanzu Valkey Operator from version 3.4.0 or 3.3.4 to 3.4.1 and updating the ValkeySentinel custom resource from Valkey 9.0.3 to 9.1.0, the Sentinel StatefulSet may continue running the previous version even though the ValkeySentinel custom resource reports the new version. The Valkey replication pods are updated automatically, but the Sentinel StatefulSet pods remain on the older image version until a manual rollout restart is performed. 

Symptoms

• ValkeySentinel custom resource shows the updated version.
• Sentinel StatefulSet pods continue running the previous version.
• Valkey replication pods are updated successfully.
• Sentinel pods do not restart automatically after the version change.
• Manual StatefulSet restart updates Sentinel pods to the expected version.

Environment

Tanzu Valkey Operator 3.4.1

Cause

The Sentinel reconciler intentionally avoids automatically rolling a healthy Sentinel quorum when a Sentinel specification change is detected.

Restarting Sentinel pods can trigger Sentinel leader re-election and may introduce the risk of an unnecessary failover of a healthy primary node. To prevent disruption of a healthy quorum, the operator does not automatically restart Sentinel StatefulSets when only the Sentinel version changes.

Resolution

This behavior is currently by design.

The Sentinel reconciler intentionally preserves a healthy Sentinel quorum and does not automatically roll the Sentinel StatefulSet after a Sentinel version update.

To apply the updated Sentinel version, perform a manual StatefulSet restart:

kubectl rollout restart statefulset <sentinel-statefulset-name> -n <namespace>

 

After the rollout completes, verify that the Sentinel pods are running the expected version.

Broadcom Engineering recognizes that updating the ValkeySentinel custom resource is generally expected to update the corresponding Sentinel instances automatically. Engineering is evaluating changes to this behavior.

This issue is targeted to be addressed in Tanzu Valkey Operator release 3.4.2.

 

For enhancement and defect tracking updates, subscribe to KB article 275360:
Subscribe to Broadcom KB articles