‘Invalid string '*' in TKG_NO_PROXY’ error when upgrade TKG cluster to TKG 1.6

Products

• VMware Tanzu Kubernetes Grid

Issue/Introduction

Symptoms:

If customer has * in TKG_NO_PROXY when installing previous TKG versions, for example, TKG 1.5.4, when they upgrade management cluster or workload cluster to TKG 1.6, they will also run into the above error.

Error: failed to update additional addon components: unable to get cluster configuration: workload cluster configuration validation failed: invalid string '*' in TKG_NO_PROXY

Cause

Symbol ‘*’ doesn’t have any effect in TKG_NO_PROXY, the correct syntax to set domain suffix should be `.vmware.com` instead of `*.vmware.com`.

Before TKG 1.6, there is no error reported but the value like `*.vmware.com` won’t have any effect. Now TKG will error out explicitly to block customer from doing this.

Resolution

Workaround: Depending on the type of cluster you are upgrading:

• Management cluster:
  1. Switch to management cluster kubectl context.
  2. Edit the configMap kapp-controller-config:
     kubectl edit cm kapp-controller-config -n tkg-system
  3. Find the data.noProxy field and change its wildcard hostname by removing *. For example, change *.vmware.com to .vmware.com
  4. Save and exit. The cluster is ready to upgrade.
• Workload cluster:
  1. Switch to workload cluster kubectl context
  2. Set environment variables for your cluster name and namespace, for example:
     CLUSTER_NAME=my-test-cluster
     NS=my-test-namespace
  3. Obtain and decode the kapp controller data values for the workload cluster:
     kubectl get secret "${CLUSTER_NAME}-kapp-controller-data-values" -n $NS -o json | jq -r '.data."values.yaml"' | base64 -d > "${CLUSTER_NAME}-${NS}-kapp-controller-data-values"
  4. Edit the ${CLUSTER_NAME}-${NS}-kapp-controller-data-values file by removing * from its kappController.config.noProxy setting. For example, change *.vmware.com to .vmware.com.
  5. Save and quit.
  6. Re-encode the data values file ${CLUSTER_NAME}-${NS}-kapp-controller-data-values:
     cat "${CLUSTER_NAME}-${NS}-kapp-controller-data-values" | base64 -w 0
  7. Edit the ${CLUSTER_NAME}-kapp-controller-data-values secret and update its data.value.yaml setting by pasting in the newly-encoded data values string.
     kubectl edit secret "${CLUSTER_NAME}-kapp-controller-data-values" -n "${NS}"
  8. Save and exit. The cluster is ready to upgrade.