Telco Cloud Automation (TCA) node pool creation fails after a long wait at "Machine object creation".

Products

• VMware
• VMware Telco Cloud Automation

Issue/Introduction

Validate the certificates (ca.crt, apiserver.crt)  the worker nodes by running the following command:
openssl x509 -in ca.crt -text -noout

If the certificates have expired, proceed with the resolution.

Symptoms:

1. Node pool creation fails after a long wait at "Machine object creation" in CAPI/TKG.
2. The VM is created in vCenter.

Environment

VMware Telco Cloud Automation 2.0

Cause

The CAPI controller's work queue can get backed up if the controllers are continuously reaching out and failing to connect to the workload cluster VIPs, due to expired certificates. This results in an eventual timeout in TCA and delays machine object creation. 

Resolution

Renew the certificates and recreate the node pools. Please refer to How to rotate certificates in a Tanzu Kubernetes Grid cluster (86251) .