Azure AD user RBAC authorization to vSphere IaaS clusters through Azure group provisioning fails after upgrade to 8.0U3 when using vCenter SSO

Products

• VMware vSphere with Tanzu

Issue/Introduction

After upgrading to 8.0U3, supervisor and guest cluster RBAC rules for users assigned permissions through Azure AD groups will not be authorized for correct RBAC permissions when using vCenter SSO.

Environment

vSphere IaaS/vSphere with Tanzu/TKGs

vCenter 8.0U3

vCenter SSO in use for authorization/authentication

Cause

WCP (Workload Control Plane) service appends an additional domain to the group names when handling the SAML token from an Azure AD user.
When a user attempts to authenticate through, wcpsvc.log instead of group names being read by WCP as groupname@domain, they are read as groupname@domain@domain. This causes RBAC issues in the provisioned supervisor and guest clusters.

Resolution

This issue is currently under investigation.

WORKAROUND: When provisioning access to these users, please temporarily add the subject for the group name to the clusterrolebinding with an additional domain appended to the end or add an additional subject that includes the group@domain@domain format.

You can typically add the following clusterrolebinding where <groupname>@<domain> represents the Azure AD group name.

- apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    annotations:
      rbac.authorization.kubernetes.io/autoupdate: "true"
    creationTimestamp: "2024-07-08T14:06:11Z"
    labels:
      kubernetes.io/bootstrapping: rbac-defaults
    name: cluster-admin
    resourceVersion: "59320952"
    uid: 2e3ff2c2-a847-425a-ba71-a36f55e07bcb
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: cluster-admin
  subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: system:masters
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: sso:[email protected]

Note: This is prefixed with "sso:" to use the vCenter SSO for authentication and authorization.

The above represents the typical clusterrolebinding configuration when this issue is not present.

The workaround requires a user to add @domain to the end of the subject name for the group OR add a subject with the groupname@domain@domain formatting.

If you append the additional @domain to the existing subject, it will appear as below:

- apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    annotations:
      rbac.authorization.kubernetes.io/autoupdate: "true"
    creationTimestamp: "2024-07-08T14:06:11Z"
    labels:
      kubernetes.io/bootstrapping: rbac-defaults
    name: cluster-admin
    resourceVersion: "59320952"
    uid: 2e3ff2c2-a847-425a-ba71-a36f55e07bcb
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: cluster-admin
  subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: system:masters
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: sso:[email protected]@example.com

OR, if you choose to add a new subject that includes the group@domain@domain format, it will appear as below:

- apiVersion: rbac.authorization.k8s.io/v1
  kind: ClusterRoleBinding
  metadata:
    annotations:
      rbac.authorization.kubernetes.io/autoupdate: "true"
    creationTimestamp: "2024-07-08T14:06:11Z"
    labels:
      kubernetes.io/bootstrapping: rbac-defaults
    name: cluster-admin
    resourceVersion: "59320952"
    uid: 2e3ff2c2-a847-425a-ba71-a36f55e07bcb
  roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: cluster-admin
  subjects:
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: system:masters
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: sso:[email protected]
  - apiGroup: rbac.authorization.k8s.io
    kind: Group
    name: sso:[email protected]@example.com

NOTE: You can edit the clusterrolebinding using the following command when logged into the correct needed context:

kubectl edit clusterrolebinding <clusterrolebindingname>

Additional Information

Note: This only affects Azure AD when used through vCenter SSO. This is not encountered when using pinniped to send authentication directly to Azure AD from vSphere IaaS cluster as an example.