Application Pod stuck in Pending ErrImagePull state due to Certificate Signed by Unknown Authority: ErrImagePull

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

A cluster's application pod is failing with ErrImagePull and an error message stating that it cannot pull the image due to an untrusted certificate authority.

An application pod can be deployed on a workload cluster to run on a workload cluster node/VM.

Workloads in vSphere Pods can be deployed on a Supervisor cluster to run on an ESXi host.

 

While connected to the cluster context where the application pod is running, the following symptoms are observed:

• The failing application pod is in Pending or ErrImagePull state:
  

  kubectl get pods -n <application namespace>

• When describing the failing application pod, an error message similar to the following is present:
  

  kubectl describe pod -n <application namespace> <application pod name>
  
  Status: Pending
  Reason: ErrImagePull
  Message: failed to pull images: failed to get images: Image <application image> has failed. Error: Failed to resolve on node <node name>. Reason: Http request failed. Code 400: ErrorType(3) failed to do request: Head "<private registry>": tls: failed to verify certificate: x509: certificate signed by unknown authority: ErrImagePull

Environment

vSphere Supervisor

This issue can occur regardless of whether or not the cluster that the application pod is running on is managed by Tanzu Mission Control (TMC)

Cause

If the application pod is within a workload cluster, the workload cluster is not configured to trust the registry's certificate.

If the application pod is a vSphere Pod deployed in the Supervisor cluster, the Supervisor cluster or ESXi hosts do not trust the registry's certificate.

Resolution

The YAML for the workload cluster will need to have the CA added to its additionalTrustedCAs section.

Please note that a change to the additionalTrustedCAs may result in a rolling redeployment of all nodes in the edited workload cluster.

See the below documentation according to the clusterClass:

ClusterClass builtin-generic

ClusterClass OsConfiguration Documentation

ClusterClass v3.4.0 YAML Example

 

ClusterClass tanzukubernetescluster / v1beta1

v1beta1 Cluster with Additional Trusted CA Certificates

Integrate VKS Clusters with a Private Container Registry

 

For deploying workloads in vSphere Pods in the Supervisor cluster and ESXi hosts, see the below documentation:

Deploying Workloads to vSphere Pods

Additional Information

Create a Pod from a Container Image in a Private Registry