TKGm - Node Pool Creation Fails despite "Success" Message

Products

• VMware Tanzu Kubernetes Grid Management

Issue/Introduction

The tanzu cluster node-pool set command returns a success message, but the new Node Pool (MachineDeployment) is not created.

tanzu cluster node-pool set <CLUSTER> -f <NODE_POOL_DEFINITION>.yaml
#> Cluster update for node pool 'md-test' completed successfully

• "kubectl get md -A" does not show the new Node Pool
• "tanzu cluster node-pool list <CLUSTER>" does not show the new Node Pool
• The cluster status remains running in "tanzu cluster list"

Environment

Tanzu Kubernetes Grid

Cause

In TKGm, creating a Node Pool is part of the Cluster Topology reconciliation process.

If the Cluster object itself fails to reconcile, it cannot create child resources like MachineDeployments.

Resolution

Identify the cause in the Cluster object status:

kubectl describe cluster <CLUSTER> -n <NAMESPACE>
kubectl get cluster <CLUSTER> -n <NAMESPACE> -o yaml

• Navigate to Status.Conditions
• If TopologyReconciled is False, the Message field will contain the error details

Example Error

If the control plane replica count is accidentally set to an even number(2 or 4), the reconciliation will fail with the following error:

Message:               error reconciling the Cluster topology: failed to create patch helper for KubeadmControlPlane/<CLUSTER>-controlplane-###: server side apply dry-run failed for modified object: admission webhook "validation.kubeadmcontrolplane.controlplane.cluster.x-k8s.io" denied the request: KubeadmControlPlane.cluster.x-k8s.io "<CLUSTER>-controlplane-###" is invalid: spec.replicas: Forbidden: cannot be an even number when etcd is stacked