TKG Cluster Upgrade Fails: No Compatible TKR Found Matching TKR Name/Prefix

Products

• VMware Tanzu Mission Control

Issue/Introduction

When attempting to upgrade a Tanzu Kubernetes Grid (TKG) workload cluster using a specific Tanzu Kubernetes Release (TKR), the upgrade fails with an error indicating that no compatible TKR was found matching the specified TKR name or prefix. The error may look similar to the following:

Error: cluster cannot be upgraded, no compatible upgrades found matching the TKR name/prefix ‘v1.28.11—vmware.2-tkg.2’, available compatible upgrades [v1.30.2—vmware.1-tkg.1]

This occurs even though the target TKR appears in the output of tanzu kubernetes-release get and is marked as compatible and active. Upon further inspection, the cluster may be running a different Kubernetes version (e.g., v1.28.7) while its tanzuKubernetesRelease label may reference a different TKR entirely (e.g., v1.29.6---vmware.1-tkg.3).

 

 

 

Cause

This issue typically occurs when there is a mismatch between the actual Kubernetes version running on the cluster and the tanzuKubernetesRelease label applied to the cluster object. The label may have been set during a previous upgrade attempt or manually edited, and it no longer reflects the cluster’s real state.

The upgrade controller uses the TKR label to determine upgrade paths. If the label does not match the current version running in the cluster, no valid upgrade path can be resolved, even if a compatible target TKR is available.

In some cases, the mismatch may also occur if the TKR OVA associated with a referenced version (e.g., v1.29.6) is missing from vCenter, causing upgrades to fail.

Resolution

• 1. Identify the actual Kubernetes version running in the cluster
     • Use the following command to verify the version currently running:
       • kubectl get nodes
       • Make sure the version displayed here matches what you expect (e.g., v1.28.7+vmware.1).
  2. Check the tanzuKubernetesRelease label applied to the cluster
     • Run the following to inspect the labels on the cluster resource:
       • kubectl get cluster <cluster> -n <namespace> --show-labels
       • Look for a label that begins with tanzuKubernetesRelease:. If it doesn’t match the version retrieved in step 1, this is likely the source of the issue.
  3. Determine the correct TKR name for the actual version
     • List all available TKRs:
       • tanzu kubernetes-release get
       • Look for the full name corresponding to the version shown in step 1 (e.g., for v1.28.7+vmware.1, the full TKR might be v1.28.7---vmware.1-tkg.1).
  4. Patch the cluster with the correct TKR label
     • Update the cluster label to reflect the correct tanzuKubernetesRelease:
       • kubectl label cluster <cluster> -n <namespace> tanzuKubernetesRelease= --overwite
       • For example:
         • kubectl label cluster <cluster> -n <namespace> tanzuKubernetesRelease=v1.28.7—vmware.1-tkg.1 --overwrite
  5. Check available upgrades again
     • Re-run the upgrade discovery:
       • tanzu cluster available-upgrades get <cluster_name>
  6. Verify that the target version (e.g., v1.28.11---vmware.2-tkg.2) is now shown as a compatible upgrade path.
  7. Retry the upgrade
     • Once the label is corrected and a valid upgrade path is confirmed, re-attempt the upgrade using your normal process.

  Notes:

  • If the previous upgrade attempt referenced a version that is not present in vCenter (e.g., the OVA was never uploaded), the cluster may have retained an invalid TKR reference.
  • This behavior is documented in Broadcom KB 374451, which provides guidance for correcting mismatched TKR metadata.

Additional Information

Resolving "no available upgrades for cluster" during upgrade of legacy TKGm cluster