Supervisor Cluster Stuck in "Configuring" State with "Authentication is required" error due to missing container registry password.

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

• The Supervisor cluster status shows as Configuring in the vSphere Client under Workload Management with below error:

 Error: Syncing directory '0': Syncing directory '.' with imgpkgBundle contents: Fetching image: Error while preparing a transport to talk with registry: Unable to create round tripper: GET https://######.######.######.######:443/artifactory/api/docker/null/v2/token?scope=repository%######%2Ftkg-service%2F3.4.0%2Btkg-service%3Apull&service=########.#######.######.com%3A443: Authentication is required. 

• The kapp-controller reports authentication errors when attempting to pull images from the configured private container registry.
• PackageInstall (pkgi) resources remain in a ReconcileFailed 
• Verified the certificate and password is provided in UI under Configure > Namespaces > Container Registries

Environment

VMware vSphere Kubernetes Service

Cause

The Kubernetes secret responsible for authenticating the Supervisor cluster to the configured Container Registry is incomplete. Because the secret lacks the required password, the kapp-controller fails to authenticate to the repository, stalling the reconciliation process Supervisor configuring state.

Note: Authentication occurs due to password or certificate issue. It is recommended to validate both password and certificate.

To verify the secret:

1. SSH to one of the control plane VM of supervisor

2. run the command: "kubectl get secret image-registry-credentials -n kube-system"

3. Decode the data section using - "echo <data section> | base64 -d"

Password section will be missing right after the username in decoded secret section.

Resolution

To resolve this issue, re-apply the registry credentials through the vSphere UI to force the regeneration of the underlying Kubernetes secret.

1. Log in to the vSphere Client.
2. Navigate to Inventory > Workload Management.
3. Select the affected Supervisor cluster.
4. Navigate to Configure > Namespaces > Container Registries.
5. Select the configured registry and re-enter the correct Password.
6. Save the configuration Supervisor configuring state.

Verification:

1. Verify the Supervisor status changes to Running in the vSphere UI.
2. Check "kubectl get secret image-registry-credentials -n kube-system" and decode it, the password section will be added after the username
3. (Optional) Access the Supervisor cluster via CLI and verify that the pkgi resources have successfully reconciled: "kubectl get pkgi -A"