Replacing Harbor registry certificates with custom root CA and TLS certificate

Products

VMware vSphere with Tanzu vSphere with Tanzu Tanzu Kubernetes Grid VMware Tanzu Kubernetes Grid Management Tanzu Mission Control

Issue/Introduction

This article will guide you to use custom root CA and TLS certificate for Harbor deployment installed as a Tanzu package on a workload cluster.

Harbor will be installed using certificates generated through cert-manager by default. However, to use custom certificates, please follow the procedure in this article.

Environment

vSphere with Tanzu
VMware Tanzu Kubernetes Grid
Harbor Registry installed as a Tanzu Package
Tanzu Mission Control

Resolution

Prerequisites:

Obtain custom PEM format certificates from your CA. Root CA Certificate and the TLS Certificate along with its key for the harbor registry. If you have an intermediate CA, The CA Certificate should have both the root CA and Intermediate CA certificate chain as best practice. If you wish to add the Intermediate to the tls.crt location, it MUST be placed below the Leaf cert.
Ensure you have 3 files, ca.crt, tls.crt and tls.key in the PEM format.

For example:
- ca.crt - the file contains CA bundle (root CA and intermediate CA if applicable)
- tls.crt - Harbor TLS certificate signed by above CA certificate
- tls.key - TLS key file
The original harbor-data-values.yaml configuration file that was used to install or update harbor package.

Certificate replacement procedure:

Validate the certificate set

# Modulus Checks - below two openssl commands against tls.crt & tls.key should report the same md5 value. 
$ openssl x509 -noout -modulus -in /path/to/tls.crt | openssl md5
$ openssl rsa -noout -modulus -in /path/to/tls.key | openssl md5

# Verify the validity of the provided TLS certificate against the associated CA certificate
$ openssl verify -CAfile /path/to/ca.crt /path/to/tls.crt
/path/to/tls.crt: OK

Edit the existing harbor-data-values.yaml which was used to install or update current running Harbor deployment and replace the content from tls.crt, tls.key and ca.crt respectively.
Certificate data should be contained to the section between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----"

Please note the four leading spaces (yaml indentation) in front of the certificate data parts when copying certificate data to the file and be very cautious to not introduce any typo or extra characters.

An example harbor-data-values file is listed below:

$ vi harbor-data-values.yaml
namespace: tanzu-system-registry
hostname: harbor.mydomain.com
port:
  https: 443
logLevel: info
tlsCertificate:
  tls.crt: |
    -----BEGIN CERTIFICATE-----
    MIIGTDCCBDSgawIBAbITewAcAWX6WccEfDQLdQACAAAfZTANBggqhkiGe90BAQsF
    aDBKMbswCQYdVQQGEdJzZzEeMAoGAfUEChMDg292MQhwCgYiVQQLEjNpY2Ekjdad
    :
    :
    aBcDeF=
    -----END CERTIFICATE-----
  tls.key: |
    -----BEGIN RSA PRIVATE KEY-----
    MIIEowIBaAKCbQEAcF7Ddu/UeWUpfKPNg8G9hA8Mi8VJj/GONkhNpll2F2tmY9ca
    nL6fsoC2vZpXYqR7q1kp6rKlZlis/TZBtPTEZDu9rBivN+9Pws362xgDRnySEszW
    :
    :
    -----END RSA PRIVATE KEY-----
  ca.crt: |
    -----BEGIN CERTIFICATE-----
    MIIG1zCCBL+gAwIBAgITZwp6rKlZlis/TZBtPTEZDu9rBp6rKlZlis/TZBtPTEZD
    ADBKMQswCQYDVQQGEwJzZzEMMAoGA1UEChMDZ292MQwwCgYDVQQLEwNpY2ExHzAd
    :
    :
    pQMyduQJUUf6IPSif1NYU8zuGsbp1Al2Nxdy
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    MIIFbzCCA1egAwIBAgIQFL7hqeEllphA4S518xNnJzANBgkqhkiG9w0BAQsFADBK
    MQswCQYDVQQGEwJzZzEMMAoGA1UEChMDZ292MQwwCgYDVQQLEwNpY2ExHzAdBgNV
    :
    :
    p2pu1ge6/xF+bCGWokxtGQW0sCim0X02y1qff1TSbACyS8nXfcPcpqOXm+PlmaTJ
    tfTY
    -----END CERTIFICATE-----
...

Update the Harbor deployment using the edited harbor-data-values.yaml file with the tanzu command

$ tanzu package installed update harbor --version INSTALLED-PACKAGE-VERSION --values-file harbor-data-values.yaml --namespace INSTALLED-PACKAGE-NAMESPACE

Example:

$ tanzu package installed update harbor --package-name harbor.tanzu.vmware.com --version 2.8.4+vmware.1-tkg.1 --values-file harbor-default-values.yaml --namespace tkg-system
 Updating package 'harbor.tanzu.vmware.com'
 Getting package install for 'harbor.tanzu.vmware.com'
...

Make sure Harbor package is reconciled successfully.
```
$ tanzu package installed list -A | grep
```

For TMC managed Harbor:

When Harbor is managed via TMC, the same certificates and ordering can be used for replacement. To update the certs, navigate to the Harbor Package in TMC, select the Edit option:
Find the Overlay YAML section:
Edit the tls.crt, tls.key, and ca.crt sections using the same certificates and process noted in the Certificate Replacement Procedure above, ensure cert ordering is correct.

Additional Information

After replacing the Harbor certificates, you may need to update the client side Harbor root CA in case the signing root CA has also been updated.
In that situations, please follow below extra procedure.

It is possible to add the Intermediate Certificate to the tls.crt, however, the cert chain will not work UNLESS the Intermediate certificate is placed BELOW the Leaf certificate. Reference the RFC documentation for details on this