Error: "You must be logged in to the server (Unauthorized)" when using TKG kubeconfig

Products

• VMware Tanzu Kubernetes Grid Management

Issue/Introduction

• When managing a Tanzu Kubernetes Grid (TKG) environment, access to a workload cluster is lost.
• Executing commands such as kubectl get nodes returns the following error: "error: You must be logged in to the server (Unauthorized)"
• Standard user contexts generated via tanzu cluster kubeconfig get <cluster-name> expire after approximately 10 hours.
• Administrator contexts generated via tanzu cluster kubeconfig get <cluster-name> --admin expire unexpectedly after several months or immediately following a cluster upgrade.

Environment

TKG 2.5.2

Cause

This is an expected security behavior of the authentication mechanisms in TKG.

• The issued access token for Standard User Access has a strict Time-To-Live (TTL), typically 10 hours, to enforce zero-trust security.
• Using Admin Access (--admin) method bypasses the token expiry duration and extracts the core Kubernetes client certificate.
• By default, these certificates have a maximum validity period of 1 year. Furthermore, when a cluster undergoes a version upgrade or an automated certificate rotation, new certificates are minted.
• Any previously downloaded --admin certificates become immediately invalid, even if their Not After expiration date is in the future.

Resolution

Depending on your access requirements, use one of the following methods to resolve the issue:

Option 1:

Restore Interactive User Access

For standard interactive access, you must re-authenticate to generate a new short-lived session token:Run the following command to log in and refresh your context:

tanzu cluster kubeconfig get <cluster-name>

Option 2:

Restore Administrator Access

If your --admin certificate expired due to time or a cluster upgrade, you must download the newly generated certificate from the management cluster.Run the following command to retrieve the updated administrator certificate:

tanzu cluster kubeconfig get <cluster-name> --admin

Option 3:

Create a Non-Expiring Token for Automation (CI/CD)

If you require persistent access that bypasses the 10-hour timeout and survives cluster upgrades, you must create a dedicated Kubernetes ServiceAccount.

1. 1. Log in to the cluster using a valid administrator context.

   2. Create a ServiceAccount and bind it to the required role (e.g., cluster-admin): kubectl create serviceaccount automation-sa -n kube-system kubectl create clusterrolebinding automation-sa-rb --serviceaccount=kube-system:automation-sa --clusterrole=cluster-admin

   3. Create a long-lived secret for the ServiceAccount. Create a file named sa-secret.yaml: