TKGi cannot get clusters from TKGI: Status: 422 Plan xxxxxx not found
search cancel

TKGi cannot get clusters from TKGI: Status: 422 Plan xxxxxx not found

book

Article ID: 313133

calendar_today

Updated On:

Products

VMware VMware vSphere with Tanzu

Issue/Introduction

Symptoms:
When trying to get tkgi clusters using TKGi CLI or TKGi Management-Console Clusters tab, Error is shown below 

Example: 
ubuntu@opsmgr-28-slot-35-tanzu-gss-labs:~$ tkgi clusters 
Error: Plan 2fa824527-318d-4253-9f8e-0025863b8c8a not found.

 Same above error is shown in TKGi management-console UI 


Environment

Tanzu Kubernetes Grid Integrated Edition 1.1.14.1
VMware Tanzu Kubernetes Grid Integrated Edition 1.x

Cause

Plan with Plan-ID "2fa824527-318d-4253-9f8e-0025863b8c8a" was set to inactive or deleted from TKGi mc after being used to create a new cluster.
The newly created cluster is still existing in a failed status but the plan was deleted/inactive and Apply Changes was applied to TKGI tile/TKGi mc before the deletion of the failed cluster was performed. 
 
ubuntu@opsmgr-28-slot-35-tanzu-gss-labs:~$ tkgi clusters 
PKS Version      Name  k8s Version  Plan Name  UUID                                  Status     Action
1.14.4-build.12  tas   1.23.14      medium     330e0b7b-2412-4bd6-a05d-1fc1f651233c  succeeded  CREATE
1.14.4-build.12  test  1.23.14      Plan-4     9b75a2de-805a-498f-a9ef-42884638dfb3  failed     CREATE



Resolution

Login to PKS DB to verify that the missing plan is existing by running steps below 
  1. SSH to PKS DB 
  2. Start the MySQL process using the following command
  3. Select pks mysql DB 
  4. Get all tables in PKS DB 
  5. Check the Cluster Name and Plan ID associated to each cluster 
Commands:
Step 1
ubuntu@opsmgr-28-slot-35-tanzu-gss-labs:~$ bosh -d pivotal-container-service-c4c6555cdc7e36081ab0 ssh pks-db/762542de-dc66-41cd-a0e2-75256a013e26

Step2
pks-db/762542de-dc66-41cd-a0e2-75256a013e26:~$ sudo -i
pks-db/762542de-dc66-41cd-a0e2-75256a013e26:~#  /var/vcap/packages/pxc/bin/mysql --defaults-file=/var/vcap/jobs/pxc-mysql/config/mylogin.cnf

Step 3
mysql> \u pks
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed

Step 4
mysql> SELECT * FROM information_schema.tables WHERE table_schema = 'pks';

mysql> SELECT * FROM information_schema.tables WHERE table_schema = 'pks';
+---------------+--------------+--------------------------------------------+------------+--------+---------+------------+------------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+---------------+
| TABLE_CATALOG | TABLE_SCHEMA | TABLE_NAME                                 | TABLE_TYPE | ENGINE | VERSION | ROW_FORMAT | TABLE_ROWS | AVG_ROW_LENGTH | DATA_LENGTH | MAX_DATA_LENGTH | INDEX_LENGTH | DATA_FREE | AUTO_INCREMENT | CREATE_TIME         | UPDATE_TIME         | CHECK_TIME | TABLE_COLLATION | CHECKSUM | CREATE_OPTIONS | TABLE_COMMENT |
+---------------+--------------+--------------------------------------------+------------+--------+---------+------------+------------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+---------------+
| def           | pks          | INT_LOCK                                   | BASE TABLE | InnoDB |      10 | Dynamic    |          1 |          16384 |       16384 |               0 |            0 |         0 |           NULL | 2023-02-14 00:52:26 | 2023-02-25 09:20:02 | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | cluster                                    | BASE TABLE | InnoDB |      10 | Dynamic    |          2 |           8192 |       16384 |               0 |       114688 |         0 |           NULL | 2023-02-14 00:52:26 | 2023-02-25 09:13:47 | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | cluster_parameters                         | BASE TABLE | InnoDB |      10 | Dynamic    |          2 |           8192 |       16384 |               0 |        16384 |         0 |           NULL | 2023-02-14 00:52:26 | 2023-02-25 09:13:41 | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | cluster_tags                               | BASE TABLE | InnoDB |      10 | Dynamic    |          0 |              0 |       16384 |               0 |        16384 |         0 |              1 | 2023-02-14 00:52:26 | NULL                | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | compute_profile                            | BASE TABLE | InnoDB |      10 | Dynamic    |          0 |              0 |       16384 |               0 |        32768 |         0 |           NULL | 2023-02-14 00:52:26 | NULL                | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | flyway_schema_history                      | BASE TABLE | InnoDB |      10 | Dynamic    |         44 |            372 |       16384 |               0 |        16384 |         0 |           NULL | 2023-02-14 00:52:26 | NULL                | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | kubernetes_profile                         | BASE TABLE | InnoDB |      10 | Dynamic    |          0 |              0 |       16384 |               0 |        32768 |         0 |           NULL | 2023-02-14 00:52:26 | NULL                | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | kubernetes_profile_customization_component | BASE TABLE | InnoDB |      10 | Dynamic    |          0 |              0 |       16384 |               0 |        16384 |         0 |           NULL | 2023-02-14 00:52:26 | NULL                | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | maintenance                                | BASE TABLE | InnoDB |      10 | Dynamic    |          0 |              0 |       16384 |               0 |            0 |         0 |              1 | 2023-02-14 00:52:26 | NULL                | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | network_profile                            | BASE TABLE | InnoDB |      10 | Dynamic    |          0 |              0 |       16384 |               0 |        32768 |         0 |           NULL | 2023-02-14 00:52:26 | NULL                | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | quota                                      | BASE TABLE | InnoDB |      10 | Dynamic    |          0 |              0 |       16384 |               0 |        32768 |         0 |           NULL | 2023-02-14 00:52:26 | NULL                | NULL       | utf8_unicode_ci |     NULL |                |               |
| def           | pks          | upgrade_task                               | BASE TABLE | InnoDB |      10 | Dynamic    |          0 |              0 |       16384 |               0 |        16384 |         0 |           NULL | 2023-02-14 00:52:26 | NULL                | NULL       | utf8_unicode_ci |     NULL |                |               |
+---------------+--------------+--------------------------------------------+------------+--------+---------+------------+------------+----------------+-------------+-----------------+--------------+-----------+----------------+---------------------+---------------------+------------+-----------------+----------+----------------+---------------+

Step 5
mysql> select NAME,PLAN_ID from cluster;
+------+---------------------------------------+
| NAME | PLAN_ID                               |
+------+---------------------------------------+
| tas  | 58375a45-17f7-4291-acf1-455bfdc8e371  |
| test | 2fa824527-318d-4253-9f8e-0025863b8c8a |
+------+---------------------------------------+

 

  • After verifying missing Plan ID which will be in this example "2fa824527-318d-4253-9f8e-0025863b8c8a"
  • Recreate the Plan manually with same exact name and Plan number. In this case it will be as follows 
Name: Plan-4 
Plan number is Plan 4 
  • Apply changes on TKGi tile in opsman or if using TKGI management-console apply changes from TKGi mc 
  • After recreating the missing plan, you will be able to fetch clusters using TKGi CLI successfully as shown below 
ubuntu@opsmgr-28-slot-35-tanzu-gss-labs:~$ tkgi clusters 

PKS Version      Name  k8s Version  Plan Name  UUID                                  Status     Action
1.14.4-build.12  tas   1.23.14      medium     330e0b7b-2412-4bd6-a05d-1fc1f651233c  succeeded  CREATE
1.14.4-build.12  test  1.23.14      Plan-4     9b75a2de-805a-498f-a9ef-42884638dfb3  failed     CREATE

ubuntu@opsmgr-28-slot-35-tanzu-gss-labs:~$ 

 


Additional Information

Plan IDs are hardcoded to plan numbers where Plan 1 named small is hardcoded to Plan ID "8A0E21A8-8072-4D80-B365-D1F502085560" 

Example: 
ubuntu@opsmgr-28-slot-35-tanzu-gss-labs:~$ tkgi plans 

Name    ID                                     Description
small   8A0E21A8-8072-4D80-B365-D1F502085560   This plan will configure a lightweight kubernetes cluster. Not recommended for production workloads.
medium  58375a45-17f7-4291-acf1-455bfdc8e371   This plan will configure a medium sized kubernetes cluster, suitable for more pods.
large   241118e5-69b2-4ef9-b47f-4d2ab071aff5   Example: This plan will configure a large kubernetes cluster for resource heavy workloads, or a high number of workloads.
Plan-4  2fa824527-318d-4253-9f8e-0025863b8c8a  Plan 4 for testing 
Plan-5  fa824527-318d-4253-9f8e-0025863b8c8a   Plan 5 testing 

ubuntu@opsmgr-28-slot-35-tanzu-gss-labs:~$ 



Impact/Risks:
After applying changes where the plan was set to inactive/deleted and still the newly created cluster present. Trying to fetch tkgi clusters will show the above error that plan xxxxxx is not found