Machine Fail to Bootstrap Due to Missing kube-config ConfigMap in VKS Cluster

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

During a rolling update, new machines fail to bootstrap. Each new VM reaches the kubeadm phase but does not generate /etc/kubernetes/pki, /var/lib/kubelet/config.yaml, or /etc/kubernetes/admin.conf. The kubelet service does not start, and the machine status remains in a waiting state for nodeRef/Node.

In the node’s bootstrap logs located at /var/log/cloud-init-output.log and /var/log/cloud-init.log, kubeadm terminates during the kubelet-start phase with the following errors:

configmap “kubelet-config” not found

error execution phase kubelet-start: failed to load kubelet configuration: configmap “kubelet-config” not found

MachineSet events show:

Machine has unhealthy node

KubeadmControlPlane (KCP) describe output shows:

waiting for control plane/worker to pass preflight checks to continue reconciliation: Machine does not have a corresponding Node yet (Machine.status.nodeRef not set)

Reviewing /var/log/cloud-init-output.log on the affected node confirms kubeadm fails immediately after attempting to write the kubelet configuration file.

Network connectivity and control-plane endpoint checks remain functional, isolating the issue to the missing ConfigMap.

This condition prevents new VMs from completing initialization and blocks the rolling update process.

Cause

The kubelet-config ConfigMap in the guest cluster’s kube-system namespace is missing.

During bootstrap, kubeadm retrieves this ConfigMap to obtain the default kubelet configuration parameters required to create /var/lib/kubelet/config.yaml. If the ConfigMap is absent, kubeadm cannot complete the kubelet-start phase and reports the error “configmap ‘kubelet-config’ not found,” leaving the node unregistered.

Resolution

1. From a healthy control-plane node in the same guest cluster, collect the kubelet configuration file located at /var/lib/kubelet/config.yaml.
2. Using the contents of that file, create a new ConfigMap in the guest cluster’s kube-system namespace named kubelet-config.
3. If multiple clusters exist with the same Kubernetes version, the kubelet-config ConfigMap from a working cluster can also be used as a reference to recreate the missing one.
4. Apply the ConfigMap and verify that both kubeadm-config and kubelet-config ConfigMaps are present in the kube-system namespace.
5. Delete the failed Machine from the Supervisor namespace so that WCP regenerates bootstrap data and creates a new control-plane VM.
6. Confirm that the new control-plane VM successfully bootstraps, generates /var/lib/kubelet/config.yaml, and registers as Ready in the cluster.