Pod with gmsa credential fails to start with error "The request is not supported"

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

• Windows gMSA webhook has been installed in VKS cluster by following the steps in Install Windows gMSA Webhook with custom values from windows-gmsa-webhook-data-values.yaml
• Pod yaml has specified the gMSA credential spec by:
  

  securityContext:
  windowsOptions:
    gmsaCredentialSpecName: "<gmsa-credential-spec-name>"
  
  Note: gMSA credential spec can be get after login VKS cluster with 'kubectl get gmsacredentialspecs.windows.k8s.io'

• Pod fails to start due to gMSA credential from the events of the Pod:
  
  kubectl describe pod -n <namespace> <pod-name>

    State:         Waiting
      Reason:      CrashLoopBackOff
    Last State:    Terminated
      Reason:      StartError
      Message:     failed to create containerd task: failed to create shim task: failed to generate container credential guard instance: hcs::ModifyServiceSettings: winapi error #3489661115
[Event Detail: Failed to setup the external credentials for Container '<container-id>': The request is not supported. (0xC00000BB). Provider: <provider-id> EventID: 11507]
      Exit Code:    128

Environment

vSphere Kubernetes Service

Cause

The hostAccountConfig in windows-gmsa-webhook-data-values.yaml is not needed which will cause the Host Compute Service (HCS) unable to handle the injection of gmsa credential.

Resolution

To resolve the issue:

1. Login VKS cluster. See Connect to a Tanzu Kubernetes Cluster as a vCenter Single Sign-On User
2. Check if gMSA credential spec existing which should have been created by applying windows-gmsa-webhook-data-values.yaml when installed gMSA webhook.
3. Backup the gMSA credential spec:
   
   kubectl get gmsacredentialspecs.windows.k8s.io -o yaml > gmsacredentialspecs.yaml
   
   
4. Delete the gMSA credential spec:
   
   kubectl delete gmsacredentialspecs.windows.k8s.io <gmsa-credential-spec-name>
   
   
5. Fetch gMSA credential spec details from Active Directory and create a new gmsacredentialspecs in VKS cluster in the Step 3 of Using gMSAs in Kubernetes
   
   Note: the values are case letter that need to strictly follow the format of gmsacredentialspecs yaml sample. 
   
   For example:
   
   
apiVersion: windows.k8s.io/v1
kind: GMSACredentialSpec
metadata:
  name: myservice
credspec:
  CmsPlugins:
  - ActiveDirectory
  DomainJoinConfig:
    Sid: <SID>
    MachineAccountName: MyService
    Guid: <AD-GUID>
    DnsTreeName: sample.com
    DnsName: sample.com
    NetBiosName: SAMPLE    # the domain need be upper case
  ActiveDirectoryConfig:
    GroupManagedServiceAccounts:
    - Name: MyService
      Scope: sddc.com
    - Name: MyService
      Scope: SAMPLE        # the domain need be upper case


6. Apply the new gmsacredentialspec in VKS cluster and update its name in Pod yaml:
   
   securityContext:
  windowsOptions:
    gmsaCredentialSpecName: "<gmsa-credential-spec-name>"
   
   
7. Run the Pod again which should run successfully.