Unable to upgrade workload cluster from TCA UI when addon is in failure status

Products

• VMware
• VMware Telco Cloud Automation

Issue/Introduction

It will impact the upgrade cluster, need one workaround to progress.

Symptoms:
In TCA UI, Failed to upgrade cluster and get error message "[registries: Fail to get password of user admin from Secret harbor-cred-kbs-f113] . You can retry to upgrade the cluster.", then In TCA UI to click retry button to upgrade the cluster, get an error message to "The version has been XXX".

Environment

VMware Telco Cloud Automation 1.9
VMware Telco Cloud Automation 1.8

Resolution

Currently, there is no solution.

Workaround:

1. SSH into TCA-CP.
2. Get cluster UUID by "kbsctl show workloadclusters" which will replace the Step3 and Step4 cluster UUID in URL.
3. Put addon again to make addon successful.

Curl -X GET "http://127.0.0.1:8888/api/v1/workloadcluster/415fd0b0-9eda-4a13-a3b7-4f9948e9ea4f/addon" > addon.json

Add harbor password to addon.json if configured harbor in the addon otherwise can skip this step. 

"externalHarborPwd": "harbor_password"

curl -X PUT "http://127.0.0.1:8888/api/v1/workloadcluster/415fd0b0-9eda-4a13-a3b7-4f9948e9ea4f/addon" -d "`cat cat addon.json`"

4. Reset cluster status to failure by "kbsctl debug set-cluster-status -i 415fd0b0-9eda-4a13-a3b7-4f9948e9ea4f --status Failure".
5. Retry upgrade from TCA UI.