NSX Application Platform is showing "Unable to connect" error due to expiry of Kubernetes Configuration.
search cancel

NSX Application Platform is showing "Unable to connect" error due to expiry of Kubernetes Configuration.

book

Article ID: 306236

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

This article provides steps to replace the default Tanzu Kubernetes Cluster configuration file with a non-expiring token after the NSX Application Platform deployment. The steps in this article apply to the NSX Application Platform deployments that were originally installed with a default kubeconfig file obtained with the command: 
cat /root/.kube/config > default_kubeconfig


Symptoms:
  • From NSX-T 3.2 with NSX Application platform (NAPP) deployed.
  • This issue will arise if NAPP is deployed on top of vSphere with Tanzu Kubernetes.
  • The following error banner is displayed when accessing NAPP:
image.png

Environment

VMware NSX-T Data Center
VMware NSX-T Data Center 3.x

Cause

The default kubeconfig file in a vSphere with Tanzu Guest Kubernetes Cluster contains a token which expires after ten hours by default, aforementioned warning for the NSX Application Platform in the NSX UI.

Resolution

To resolve this issue the vSphere with Tanzu Kubernetes tokens needs to be recreated.

1. Log in the vSphere with Tanzu Guest Kubernetes cluster

kubectl vsphere login --server <supervisor-cluster_ip> -u <user> --tanzu-kubernetes-cluster-name <tk-cluster-name> --tanzu-kubernetes-cluster-namespace <namespace>
The <supervisor-cluster_ip> parameter is the Control Plane Node Address which can be found in the vSphere Client by selecting Workload Management > Supervisor Clusters.
The parameter <tk-cluster-name> is the name of the Tanzu Kubernetes cluster and the <namespace> parameter is the vSphere namespace where this cluster resides.

Example command with parameter values:

kubectl vsphere login --server #.#.#.# -u [email protected] --tanzu-kubernetes-cluster-name napp-tkg-cluster --tanzu-kubernetes-cluster-namespace napp
2. Generate an administrator service account and create a cluster role binding:
kubectl create serviceaccount napp-admin -n kube-system

kubectl create clusterrolebinding napp-admin --serviceaccount=kube-system:napp-admin --clusterrole=cluster-admin
3. Obtain the authentication token for the administrator service account and the cluster certificate authority.
SECRET=$(kubectl get serviceaccount napp-admin -n kube-system -ojsonpath='{.secrets[].name}')

TOKEN=$(kubectl get secret $SECRET -n kube-system -ojsonpath='{.data.token}' | base64 -d)

kubectl get secrets $SECRET -n kube-system -o jsonpath='{.data.ca\.crt}' | base64 -d > ./ca.crt
4. Get the Tanzu Kubernetes Cluster URL.
CONTEXT=$(kubectl config view -o jsonpath='{.current-context}')

CLUSTER=$(kubectl config view -o jsonpath='{.contexts[?(@.name == "'"$CONTEXT"'")].context.cluster}')

URL=$(kubectl config view -o jsonpath='{.clusters[?(@.name == "'"$CLUSTER"'")].cluster.server}')


5. Retrieve the context name, cluster name and username from the existing NSX Application Platform deployment from the NSX API.

You can use the curl command or a RESTAPI client such as Postman to access the NSX API.Use the following API call: 

GET https://<manager-ip>/policy/api/v1/infra/sites/default/napp/deployment/kubeconfig
Example of API call using curl:
curl --insecure --user <admin-user> https://<manager-ip>/policy/api/v1/infra/sites/default/napp/deployment/kubeconfig

Sample output:
{
"bundle_id": "1638527923839",
"name": "default_kubeconfig",
"user": "wcp:#.#.#.#:[email protected]",
"current_context": "napp-tkg-cluster",
"cluster": "#.#.#.#",
"expiry": 1638563458000,
"connection_status": true,
"resource_type": "KubeconfigInfo",
"id": "kubeconfig",
"display_name": "kubeconfig",
"path": "/infra/sites/default/cloud-native/deployment/kubeconfig/",
"relative_path": "kubeconfig",
"parent_path": "/infra/sites/default",
"unique_id": "6210d4b1-6517-400d-####-###########",
"realization_id": "6210d4b1-6517-400d-####-###########",
"marked_for_delete": false,
"overridden": false,
"_create_time": 1638527923967,
"_create_user": "admin",
"_last_modified_time": 1638527923967,
"_last_modified_user": "admin",
"_system_owned": false,
"_protection": "NOT_PROTECTED",
"_revision": 0
}
6. Generate a configuration file for the Tanzu Kubernetes Cluster with a non-expiring token.

TO_BE_CREATED_KUBECONFIG_FILE="<file-name>"

CLUSTER_NAME="<cluster-API-output>"

USER_NAME="<user-API-output>"

CONTEXT_NAME="<current_context-API-output>"
  • The parameter <file-name> is the name of config file you wish to create.
  • The parameter <cluster-API-output> refers to the cluster value obtained as result of executing Step 5. In the example above this value is "#.#.#.#".
  • The parameter <user-API-output> refers to the user value obtained as result of executing Step 5. In the example above this value is "wcp:#.#.#.#:[email protected]".
  • The parameter <current_context-API-output> refers to the current_context value obtained as result of executing Step 5. In the example above this value is "napp-tkg-cluster".
kubectl config --kubeconfig=$TO_BE_CREATED_KUBECONFIG_FILE set-cluster $CLUSTER_NAME --server=$URL --certificate-authority=./ca.crt --embed-certs=true

kubectl config --kubeconfig=$TO_BE_CREATED_KUBECONFIG_FILE set-credentials $USER_NAME --token=$TOKEN

kubectl config --kubeconfig=$TO_BE_CREATED_KUBECONFIG_FILE set-context $CONTEXT_NAME --cluster=$CLUSTER_NAME --user=$USER_NAME

kubectl config --kubeconfig=$TO_BE_CREATED_KUBECONFIG_FILE use-context $CONTEXT_NAME
7. (Optional) The ca.crt is a temporary file created during the generation of the kubeconfig file. You can safely delete this file from your environment if you wish to do so.


8. Upload the newly generated configuration file with a non-expiring token to the NSX Application Platform.

a. On the NSX UI Home page, navigate to System > NSX Application Platform.

b. From the Actions drop-down menu, select Settings.

c. In the Configuration section, click the EDIT link.

d. In the Upload File field, click SELECT, select the newly generated config file with a non-expiry token, click Open, and click Upload.

e. Ensure that the file is successfully uploaded and click NEXT.

f. Click the RUN PRECHECKS button, to ensure the environment still satisfies all the NSX Application Platform deployment requirements.

g. Click NEXT to continue.

h. Click CLOSE.




Additional Information

Impact/Risks:
This warning does not have an impact on the functionality of the NSX Application Platform nor any of the security features currently installed.

However, you will need to generate a valid (not expired token) every time you want to:

  • Deploy the NSX Application Platform
  • Upgrade the NSX Application Platform
  • Delete the NSX Application Platform


Powered by Wolken Software