Reconciling certificate error while adding using harbor to deploy TMC Self-Managed

Products

• Tanzu Mission Control

Issue/Introduction

This KB article is intended to help troubleshoot issues when using a private harbor for a TMC repository while deploying TMC Self-Managed

Symptoms:

 tanzu package repository add tanzu-mission-control-packages --url "harbor.yourdomain.com/tmc-sm/package-repository:1.0.0" --namespace tmc-local

Waiting for package repository to be updated

Waiting for package repository reconciliation for 'tanzu-mission-control-packages'
           Fetch failed 
            | vendir: Error: Syncing directory '0':
            |   Syncing directory '.' with imgpkgBundle contents:
            |     Imgpkg: exit status 1 (stderr: imgpkg: Error: Fetching image:
            |   Error while preparing a transport to talk with the registry:
            |     Unable to create round tripper:
            |       Get "https://harbor.yourdomain.com/v2/": x509: certificate signed by unknown authority
            | )
            | 
            | Fetching resources: Error (see .status.usefulErrorMessage for details)
Error: Reconciling repository: Fetch failed

Cause

The Kapp controller configuration is not updated with the CA Harbor certificate for the private registry.

 

For vSphere 7.X, this needs to be manually added to the kapp-controller secret "kapp-controller-config".

 

For vSphere 8.0u2 and lower 8.X, this needs to be manually added to the KappControllerConfig for the corresponding vSphere Kubernetes Cluster.

 

For vSphere 8.0u3, this should use the private container registry feature as detailed in this documentation: https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-supervisor/8-0/vsphere-supervisor-services-and-workloads-8-0/deploying-supervisor-services-from-a-private-container-image-registry.html

Resolution

For TKC deployed on vCenter 7:
Edit the Kapp-controller secret with the Harbor CA certificate as below:

apiVersion: v1
kind: Secret
metadata:
  name: kapp-controller-config
  namespace: tkg-system
  annotations:
    kapp.k14s.io/change-group: apps.kappctrl.k14s.io/kapp-controller-config
data:
  caCerts: |-
    -----BEGIN CERTIFICATE-----
    xxx
    -----END CERTIFICATE-----
  httpProxy: ""
  httpsProxy: ""
  noProxy: ""
  dangerousSkipTLSVerify: ""

For TKC deployed on vCenter 8:

Starting in vSphere 8.0u3, the private container registry feature should be used instead of modifying the KappControllerConfig for private registries:

• Please follow the below documentation:
  • https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere-supervisor/8-0/vsphere-supervisor-services-and-workloads-8-0/deploying-supervisor-services-from-a-private-container-image-registry.html

 

For vSphere 8.0u2 and lower 8.X environments:

apiVersion: run.tanzu.vmware.com/v1alpha3
kind: KappControllerConfig
metadata:
  ...
  name: wc-tmc-kapp-controller-package
  namespace: testns
  ...
  resourceVersion: "1891769"
  uid: 48169c91-4f15-4a85-a8d5-116ff7d549d9
spec:
  kappController:
    config:
      caCerts: |-
        -----BEGIN CERTIFICATE-----
        xxx
        -----END CERTIFICATE-----
    createNamespace: false
    deployment:
      ...
    globalNamespace: tkg-system
  namespace: tkg-system
status:
  secretRef: wc-tmc-kapp-controller-data-values

Additional Information

Prepare cluster to host Tanzu Mission Control Self-Managed

Impact/Risks:
Adding Tanzu Mission Control repository is stuck. Hence cannot deploy Tanzu Mission Control Self-Managed