VMware vSphere Kubernetes Service upgrade fails with x509 certificate signed by unknown authority in upgrade-compatibility-service

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

• VMware vSphere Kubernetes Service upgrade fail during the pre-upgrade compatibility check. The operation halts, and the event log displays the following explicit error:

Cannot complete the operation. See the event log for details. error when querying svc-tkg-domain-c#####.svc.cluster.local:80/ucs/v2/compatibility": tls: failed to verify certificate: x509: certificate signed by unknown authority

• Additionally, inspecting the upgrade-compatibility-service pod reveals it is unresponsive and not actively writing to its logs.

Environment

VMware vSphere Kubernetes Service

Cause

The upgrade-compatibility-service pod enters a stale or unresponsive state, causing it to serve an invalid or expired internal TLS certificate during the upgrade compatibility query.

Resolution

To resolve this issue, force the upgrade-compatibility-service to re-initialize and mount its internal TLS certificates correctly:

1. Log in to the Supervisor Cluster context using the kubectl vsphere login command.

2. Delete the stale upgrade-compatibility-service pod to force a restart: kubectl delete pod -l app=upgrade-compatibility-service -n svc-tkg-domain-c#####

3. Verify that the new upgrade-compatibility-service pod reaches the Running state and is actively writing logs.

4. Proceed to retry the VMware vSphere Kubernetes Cluster upgrade operation via the UI or CLI.