VKS Cluster Management Config Error: The Supervisor is Unhealthy

Products

VMware vSphere Kubernetes Service

Issue/Introduction

When configuring VKS Cluster Management service in VCF-A 9.0.1 to connect to a Supervisor instance, an error is seen indicating the supervisor is unhealthy.

- The Supervisor Management Proxy service is configured correctly (See: Configuring the Supervisor Management Proxy)
- The Supervisor Cluster is added to VCF and assigned the appropriate region (See: Understanding Supervisors, Regions, Zones & vSphere Namespaces)
- The error in the GUI for the auto-attach-service is similar to:
Reason: ReconcileFailed. Message: vendir: Error: Syncing directory '0': Syncing directory '.' with imgpkgBundle contents: Fetching image: Error while preparing a transport to talk with the registry: Unable to create round tripper: Get "https://mgmt-image-proxy.kube-system.svc.cluster.local/v2/": Bad Gateway; GET http://mgmt-image-proxy.kube-system.svc.cluster.local/v2/: unexpected status code 502 Bad Gateway: <html><head><title>502 Bad Gateway</title></head> <body><h1>DNS error</h1> <p>DNS error (the host name of the page you are looking for does not exist)<br><br>Please check that the host name has been spelled correctly.<br></p> <!--<PROXY-NAME-></body></html> .

- The PROXY-NAME in the error above is the same proxy configured for the vCenter appliance
- The same error above is found when accessing the Supervisor via shell and running the following "kubectl get packageinstall -n vmware-system-supervisor-services svc-auto-attach.vksm.broadcom.com"

status:
  conditions:
  - message: Error (see .status.usefulErrorMessage for details)
    status: "True"
    type: ReconcileFailed
  friendlyDescription: 'Reconcile failed: Error (see .status.usefulErrorMessage for
    details)'
  lastAttemptedVersion: 0.1.0
  observedGeneration: 1
  usefulErrorMessage: "vendir: Error: Syncing directory '0':\n  Syncing directory
    '.' with imgpkgBundle contents:\n    Fetching image:\n      Error while preparing
    a transport to talk with the registry:\n        Unable to create round tripper:\n
    \         Get \"https://mgmt-image-proxy.kube-system.svc.cluster.local/v2/\":\n
    \           Bad Gateway; GET http://mgmt-image-proxy.kube-system.svc.cluster.local/v2/:
    unexpected status code 502 Bad Gateway: <html><head><title>502 Bad Gateway</title></head>\r\n<body><h1>DNS
    error</h1>\r\n<p>DNS error (the host name of the page you are looking for does
    not exist)<br><br>Please check that the host name has been spelled correctly.<br></p>\r\n<!--<PROXY-NAME--></body></html>\r\n\n"
  version: 0.1.0

Environment

VCF 9.0
vSphere with Tanzu 9.0

Cause

The supervisor cluster is configured to inherit proxy settings from the vCenter by default.
In this instance, the supervisor is unable to reach the image repository through the configured proxy and pull the auto-attach package.

Resolution

Update the supervisor proxy settings to exclude mgmt-image-proxy.kube-system.svc.cluster.local from the proxy routing.
See the following documentation for updating the proxy configuration: Configuring HTTP Proxy Settings in vSphere Supervisor