Cluster upgrade stuck in CreateStalled state

Products

• VMware Tanzu Kubernetes Grid Management

Issue/Introduction

• Upgrading the kubernetes nodes stalls.
• The output of tanzu cluster get <CLUSTER> shows the status 'createStalled'.
• The VIP is not assigned to any control plane nodes.
• The serverPool in NSX-ALB shows incorrect IPs configured.

Environment

2.5.4

Cause

The NSX-ALB certificate has expired, causing the ako pod to enter a CrashLoopBackoff state. With this pod down, updates to the serverPool cannot happen automatically and results in the misconfiguration.

Resolution

Workaround:

• Manually update the serverPool to contain the correct IPs of the cluster's control plane nodes.

Resolution:

• Renew/replace the certificate that has expired in NSX-ALB using KB Renewing expired cert.