How to unlock VM users in VKS cluster nodes

Products

VMware vSphere Kubernetes Service

Issue/Introduction

User accounts on VKS cluster nodes will be locked after multiple failed login attempts due to STIG-enforced rule in VKr.
This document outlines the steps to unlock a user account when it becomes locked due to repeated incorrect password entries.

Environment

VMware vSphere Kubernetes Service

Cause

User accounts on VKS cluster nodes will be locked if multiple failed login attempts are made due to STIG-enforced rule in VKr.

Resolution

1. Create a .yaml file called unlock_vmware-user.yaml as below

unlock_vmware_user.yaml:

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: unlock-vmware-user
  namespace: kube-system
spec:
  selector:
    matchLabels:
      app: unlock-vmware-user
  template:
    metadata:
      labels:
        app: unlock-vmware-user
    spec:
      hostNetwork: true
      volumes:
        - name: hostfs
          hostPath:
            path: /
      initContainers:
        - name: unlock-sudo
          image: photon:5.0
          command:
            - /bin/sh
            - -xc
            - |
              echo "Checking faillock for vmware-system-user..."
              chroot /host faillock --user vmware-system-user || true
              echo "Resetting faillock for vmware-system-user..."
              chroot /host faillock --user vmware-system-user --reset
              echo "Done"
              chroot /host faillock --user vmware-system-user
          securityContext:
            privileged: true
            runAsUser: 0
            runAsGroup: 0
            allowPrivilegeEscalation: true
            capabilities:
              add: ["ALL"]
          volumeMounts:
            - name: hostfs
              mountPath: /host
      containers:
        - name: sleep
          image: localhost:5000/vmware.io/pause:3.10
          securityContext:
            runAsNonRoot: true
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]
            seccompProfile:
              type: RuntimeDefault
      tolerations:
        - key: node-role.kubernetes.io/master
          operator: Exists
          effect: NoSchedule
        - key: node-role.kubernetes.io/control-plane
          operator: Exists
          effect: NoSchedule
        - key: CriticalAddonsOnly
          operator: Exists
        - key: node.alpha.kubernetes.io/notReady
          operator: Exists
          effect: NoExecute
        - key: node.alpha.kubernetes.io/unreachable
          operator: Exists
          effect: NoExecute

This will create a yaml file called unlock_vmware_user.yaml which we can apply to the workload cluster, also known as a Guest Cluster.
As per the YAML file above, this daemonset creates a pod on each node in the workload cluster and runs a few commands to unlock the vmware-system-user.
This daemonset and its pods will persist through workload cluster upgrades to prevent vmware-system-user from expiring but may require a pause image version change.
vmware-system-user is VMware by Broadcom Support's system user for troubleshooting workload clusters.

2. Use the kubectl vsphere login command to log into your workload cluster as per either of the following documentation:

3. Apply the Daemonset .yaml created in the Step 1 above.

kubectl apply -f  unlock_vmware-user.yaml

4. Confirm that the daemonset shows a total count of Ready daemonsets equivalent to the total number of nodes in the environment:
Note: The cluster-admin daemonset is often created in the default namespace

kubectl get ds unlock_vmware-user -n kube-system

Additional Information

Notes:

If a custom value for user.user (lather than vmware-system-user) has been configured, adjust the commands to use that username.
Check a related KB also : https://knowledge.broadcom.com/external/article?legacyId=90469
Japanese version: VKS クラスタノードで VM ユーザーのロックを解除する方法