CaaS Cluster Node Pool Creation /customization Stuck in processing state

Products

• VMware Telco Cloud Automation
• VMware Telco Cloud Platform

Issue/Introduction

• Node Pool is stuck in creation state
• One or More nodes are in Ready,SchedulingDisabled state as shown in the example:
  
• When deploying a cluster or nodepool with a name that was used before in the environment at some point, you will see the cluster is provisioning state.
• The customization of the nodes can also be stuck in provisioning

Environment

TCA:  3.2, 3.2.0.1, 3.3.24731888, 3.3.0.1

TCP:  5.0, 5.0.1, 5.0.2

 

Cause

• When clusters are deleted, policies are left over.
• When clusters are recreated with the same name, multiple policies are pointed at the same workload cluster and same nodepool have different clusterIDs.
• Attempts to restart CaaS Spoke will reconcile the policies in parallel.

Resolution

Resolved in TCA 3.3.0.1

 

 

Workaround:

For 3.2: Apply the Patch Tool for TCA 3.2.0.1 KB.  

For 3.3: Perform the following steps:

Prerequisites:

• TCA environment (TCA-M/TCA CP) must be patched with 3.3.0 patch 1.

Patch the TCA-M

1. Download the 330-kb-bundle.zip patch bundle:
   1. curl -LO https://packages.broadcom.com/artifactory/tca-distro/kb/3.3.0/kb-bundle/330-kb-bundle.zip
2. Copy the patch bundle over to the TCA-M
3. Unzip the patch bundle as the root user:
   1. ssh admin@<TCA-M_IP> 
   2. su -
   3. unzip 330-kb-bundle.zip -d 330-kb
   4. cd 330-kb
4. List all KBs supported by the patch script
   1. bash caas_support.sh patch list 
   2. The patch list will show all the KB supported by the script. Currently only 1 KB is needed/supported.
5. Check KB status
   1. bash caas_support.sh patch check
      

      Possible replies:

      KB status	Comments
      NotNeeded	KB is not needed on the current cluster.
      NotApplied	KB is needed and not applied.
      Applied	KB was applied successfully. When you execute apply, it will report as Skipped
      Unknown	Can not tell the status

6. Apply the patch
   1. bash caas_support.sh patch apply
      

      It could return following status:

      KB status	Comments
      NotNeeded	KB is not needed on the current cluster.
      Skipped	KB is already applied, skip it here.
      Succeed	KB is applied successfully. When you execute apply again, it will report as Skipped
      Failed	KB application failed.

7. After the TCA-M has been patch. proceed in patching the TCA-CP using the same steps used to patch the TCA-M.