LDAP group based users unable to access Kubernetes clusters in TKGI with “tkgi get-kubeconfig”

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

When using a RoleBinding in VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) that is tied to a group instead of a specific user, and users in the group attempt to get their kubeconfig access credentials through “tkgi get-kubeconfig”, you are not able to access Kubernetes clusters with the permissions defined in the Role.

Environment

Cause

Looking at the token that gets inserted in the kubeconfig, you can see that there no "roles" entry with the group information. The problem is that the kubeconfig is built with User Account and Authentication's (UAA’s) "access-token" instead of the "id-token".

Sample “access_token” payload data:

{
  "jti": "d637bd014############a99d0f9ad",
  "sub": "7ab152ee-####-####-b73c-2b65aae82c7d",
  "scope": [
    "openid",
    "roles"
  ],
  "client_id": "pks_cluster_client",
  "cid": "pks_cluster_client",
  "azp": "pks_cluster_client",
  "grant_type": "password",
  "user_id": "7ab152ee-#####-####-b73c-2b65aae82c7d",
  "origin": "ldap",
  "user_name": "#####",
  "email": "#####@######.##",
  "auth_time": 1556219667,
  "rev_sig": "ad####eab",
  "iat": 1556219668,
  "exp": 1556262868,
  "iss": "https://api.###.####-###.###.######.##:8443/oauth/token",
  "zid": "uaa",
  "aud": [
    "openid",
    "pks_cluster_client"
  ]
}

Sample “id_token” payload data:

{
  "sub": "7ab152ee-####-####-b73c-2b65aae82c7d",
  "aud": [
    "pks_cluster_client"
  ],
  "iss": "https://###.###.####-###.###.######.##:8443/oauth/token",
  "exp": 1556262868,
  "iat": 1556219668,
  "amr": [
    "ext",
    "pwd"
  ],
  "azp": "pks_cluster_client",
  "roles": [
    "developers"
  ],
  "scope": [
    "openid"
  ],
  "email": "#####@#######.##",
  "zid": "uaa",
  "origin": "ldap",
  "jti": "d637bd014############a99d0f9ad",
  "previous_logon_time": 1556138378088,
  "email_verified": false,
  "client_id": "pks_cluster_client",
  "cid": "pks_cluster_client",
  "grant_type": "password",
  "user_name": "#####",
  "rev_sig": "ad94feab",
  "user_id": "7ab152ee-#####-#####-b73c-2b65aae82c7d",
  "auth_time": 1556219667
}

UAA is able to authenticate with both, but the "access-token" does not include the authorized roles needed for Role-Based Access Control (RBAC) in Kubernetes to function properly with groups.

This affects both the “tkgi get-credentials” and “tkgi get-kubeconfig” commands.

Resolution

After running tkgi get-kubeconfig as per the "Obtain Cluster Access as a User" section of the documentation, you will need to fix the token in the kubeconfig. To do so, follow these steps:

1. Run the following command for a given user and extract the “id_token” from the json response:

curl 'https://PKS-API:8443/oauth/token' -k -XPOST -H 'Accept: application/json' -d "client_id=pks_cluster_client&client_secret=""&grant_type=password&username=UAA-USERNAME&response_type=id_token" --data-urlencode password=UAA-PASSWORD

Where:

• PKS-API is the FQDN you use to access the PKS API.

• UAA-USERNAME is the Kubernetes end user’s UAA username.

• UAA-PASSWORD is the Kubernetes end user’s UAA password.

2. Edit the kubeconfig in the $HOME/.kube/config directory.

• Locate the configuration of your user.
• Replace the users.user.auth-provider.config.id-token value with the new one obtained from the User Account and Authentication CLI (UAAC) command.
• Save the file.
• The user should now be able to access the Kubernetes cluster through kubectl to the extent that it has been granted through RoleBindings, either user or group based ones.