Upgrading a cluster to TKGi v1.19.0 fails with vsphere-cloud-controller-manager error. The bosh task logs show the following error:
Task 1973 | 12:09:01 | L starting jobs: master/xxx12312-exxx-xxxa-xxx1-xxxcaa4c0xxx (0) (canary) (00:16:30)
L Error: 'master/xxx12312-exxx-xxxa-xxx1-xxxcaa4c0xxx (0)' is not running after update. Review logs for failed jobs: vsphere-cloud-controller-manager
The vsphere-cloud-controller-manager logs show the following:
I0625 12:08:32.779718 13205 main.go:161] vsphere-cloud-controller-manager version: v1.28.0
W0625 12:08:32.779808 13205 config.go:69] ReadCPIConfigYAML failed: yaml: unknown anchor 'xI' referenced
E0625 12:08:32.779832 13205 config.go:73] ReadConfigINI failed: 1:1: expected section header
F0625 12:08:32.779844 13205 main.go:268] Cloud provider could not be initialized: could not init cloud provider "vsphere": 1:1: expected section header
Tanzu Kubernetes Grid Integrated Edition v1.19.0
This is caused by a known issue in TKGi v1.19.0, wherein the vSphere password that is having special characters is not enclosed in double quotes in the vsphere.conf file.
See the following doc that mentions this issue.
This issue is fixed in TKGi v1.19.1. The resolution is to upgrade to TKGi v1.19.1, and then upgrade the cluster again.