TKG Management cluster upgrade fails, config-values secret not found in tkg-system namespace
search cancel

TKG Management cluster upgrade fails, config-values secret not found in tkg-system namespace

book

Article ID: 313095

calendar_today

Updated On:

Products

VMware Telco Cloud Automation

Issue/Introduction

Symptoms:
TKG upgrade fails as it cant find the config-values secret for the Management cluster.
upgrade_region.go:230] tkg-pkg-tkg-system-values secret in tkg-system namespace not found, trying to fetch <Cluster Name>-config-values secret instead... 
clusterclient.go:1412] waiting for resource <Cluster Name>-config-values of type *v1.Secret to be up and running 
poller.go:63] secrets “<Cluster Name>-config-values” not found, retrying


Cause

The <Cluster Name>-config-values secret has been removed from tkg-system namespace.

Resolution

The following procedure can be used to restore the <Cluster Name>-config-values secret.
  • SSH to the jumpbox/TCA CP node that was used to create the Management cluster and locate cluster file used to create the cluster. It it normally in ~/.config/tanzu/tkg/clusterconfigs
 
  • Take a copy of the file (eg cluster-config.yaml) and remove references to providers, release, images and cert-manager leaving just Configuration variables as described here 
 
  • Create the secret yaml file 
kubectl create secret generic <Cluster-name>-config-values --from-file=value=cluster-config.yaml -n tkg-system --dry-run -o yaml > <Cluster-Name>-config-values.yaml
 
  • Update <Cluster-name>-config-values.yaml, add the following metadata and type
metadata:
  labels:
    clusterctl.cluster.x-k8s.io/move: ""
    tkg.tanzu.vmware.com/cluster-name: <Cluster Name>
  name: <Cluster Name>-config-values
  namespace: tkg-system
type: addons.cluster.x-k8s.io/resource-set
 
  • Create the secret in the Management cluster
kubectl config use-context <Mgmt Cluster Context>
kubectl apply -f <Cluster Name>-config-values.yaml
  • Initiate Management cluster upgrade again