TMC Proxy environment showing clusters as Disconnected and Updating state with ImagePullBackOff status on TMC pods

Products

• VMware Tanzu Mission Control

Issue/Introduction

You are seeing clusters in TMC with the below status

After an investigation you can see TMC pods with ImagePullBackOff status as well as below errors:

Warning Failed 11m (x4 over 13m) kubelet Failed to pull image "extensions.stacks.<company_name>.tmc.tanzu.broadcom.com/extensions/tmc-observer/tmc-observer@sha256:15187773398d54d4495ff995bbf0e81d41de53b1e856ff4a1bf7e632c71c9089": rpc error: code = Unknown desc = failed to pull and unpack image "extensions.stacks.<company_name>.tmc.tanzu.broadcom.com/extensions/tmc-observer/tmc-observer@sha256:15187773398d54d4495ff995bbf0e81d41de53b1e856ff4a1bf7e632c71c9089": failed to resolve reference "extensions.stacks.<company_name>.tmc.tanzu.broadcom.com/extensions/tmc-observer/tmc-observer@sha256:15187773398d54d4495ff995bbf0e81d41de53b1e856ff4a1bf7e632c71c9089": failed to do request: Head "https://extensions.stacks.<company_name>.tmc.tanzu.broadcom.com/v2/extensions/tmc-observer/tmc-observer/manifests/sha256:15187773398d54d4495ff995bbf0e81d41de53b1e856ff4a1bf7e632c71c9089": x509: certificate signed by unknown authority

Environment

TMC Saas
TKGs

Cause

TMC configured proxy was not been configured to allow for the image location FQDN - extensions.stacks.<company_name>.tmc.tanzu.broadcom.com

Resolution

Tanzu Mission Control, Cluster and Agent Extensions must have permission to communicate with *.tmc.tanzu.broadcom.com.

Simply allow for *.tmc.tanzu.broadcom.com via your proxy to allow for the pods/worker nodes to pull images from this image location.

As per the ports site - https://ports.broadcom.com/home/Tanzu-Mission-Control

* note at time of writing this article the link above displayed*.tmc.cloud.vmware.com. *.tmc.cloud.vmware.com is the old FQDN which was later migrated to *.tmc.tanzu.broadcom.com.