How to add private/enterprise Certificate Authority to Tanzu Application Platform components

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

How to add private/custom Certificate Authority (CA) to the following Tanzu Application Platform (TAP) components.

• BuildService
• Convention-Controller
• OOTB-templates
• Source-Controller

Environment

Product Version: 1.0

Resolution

1. BuildService - this component expose the ability to provide CA certificates as part of the "tap-values.yml" under the buildservice.ca_cert_data tag.  
 

# tap-values.yml (snippet)

buildservice:
  ca_cert_data: |
    -----BEGIN CERTIFICATE-----
    MIIGPjCCBCagAwIBAgIJAKMduaqpCYfYMA0GCSqGSIb3DQEBCwUAMIGrMQswCQYD
    .....
    J3S9Zf4F50HbrhncESiTXyXW
    -----END CERTIFICATE-----

2. Other components do not expose through TAP nor even directly at this time. The following components require YTT overlays deployments to access to registry with custom CA.

• conventions-controller
• source-controller
• ootb-templates

Notes:

1. This workaround applies to TAP version 1.0.x
2.  Kubelet access to the registry requires the TKGI platform-operator to install the custom CA at the BOSH level (OpsManager "director config" tile)

Steps on how to execute YTT Merge-Overlay

YTT Overlays with inlined custom CA are stored in generic K8S secret objects in the tap-install namespace.

The "tap" PackageInstall is then patched to support a reference to each of these overlays secrets in the following annotation:

ext.packaging.carvel.dev/ytt-paths-from-secret-name.<index>: <overlay-secret-object>

a. Patching ‘conventions-controller’

Update ‘conventions-overlay.yaml’ to include your custom CA and save to file.
 

# filename: conventions-overlay.yaml

#@ load("@ytt:overlay", "overlay")

apiVersion: v1
kind: Secret
metadata:
  name: patch-convention-controller-ca-cert
  namespace: tap-install
stringData:
  patch.yaml: |
    #@ load("@ytt:overlay", "overlay")

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: ca-cert
      namespace: conventions-system
    data:
      custom-ca.crt: |
                -----BEGIN CERTIFICATE-----
                MIIDgDCCAmigAwIBAgIVAK9B4jDo/4HiN5htf9vWxl9GS3AfMA0GCSqGSIb3DQEB
                …
                HpgVnpRVUraH2lL7XAewZMZDgi36TBCx6MFEz+KbRtaXrQXZ
                -----END CERTIFICATE-----


    #@overlay/match by=overlay.subset({"kind":"Deployment","metadata":{"name":"conventions-controller-manager"}})
    ---
    spec:
      template:
        spec:
          containers:
          #@overlay/match by=overlay.subset({"name": "manager"})
          - volumeMounts:
            #@overlay/append
            - name: ca-cert
              mountPath: /etc/ssl/certs/custom-ca.crt
              subPath: custom-ca.crt
          volumes:
          #@overlay/append
          - name: ca-cert
            configMap:
              name: ca-cert

#@overlay/match by=overlay.subset({"kind":"PackageInstall","metadata":{"name":"conventions-controller"}})
---
metadata:
  #@overlay/match missing_ok=True
  annotations:
    #@overlay/match missing_ok=True
    ext.packaging.carvel.dev/ytt-paths-from-secret-name.0: patch-convention-controller-ca-cert

Create K8s Secret to store content

kubectl create secret generic -n tap-install convention-service-cert --from-file=./conventions-overlay.yml

b. Patching ‘Source-Controller’

Update ‘source-overlay.yaml’ to include your custom CA and save to file

# filename: sources-overlay.yaml


#@ load("@ytt:overlay", "overlay")

apiVersion: v1
kind: Secret
metadata:
  name: patch-source-controller-ca-cert
  namespace: tap-install
stringData:
  patch.yaml: |
    #@ load("@ytt:overlay", "overlay")

    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: ca-cert
      namespace: source-system
    data:
      custom-ca.crt: |
                -----BEGIN CERTIFICATE-----
                MIIDgDCCAmigAwIBAgIV…
                -----END CERTIFICATE-----


    #@overlay/match by=overlay.subset({"kind":"Deployment","metadata":{"name":"source-controller-manager"}})
    ---
    spec:
      template:
        spec:
          containers:
          #@overlay/match by=overlay.subset({"name": "manager"})
          - volumeMounts:
            #@overlay/append
            - name: ca-cert
              mountPath: /etc/ssl/certs/custom-ca.crt
              subPath: custom-ca.crt
          volumes:
          #@overlay/append
          - name: ca-cert
            configMap:
              name: ca-cert

#@overlay/match by=overlay.subset({"kind":"PackageInstall","metadata":{"name":"source-controller"}})
---
metadata:
  #@overlay/match missing_ok=True
  annotations:
    #@overlay/match missing_ok=True
    ext.packaging.carvel.dev/ytt-paths-from-secret-name.2: patch-source-controller-ca-cert

Create K8s Secret to store content

kubectl create secret generic -n tap-install source-service-cert --from-file=./sources-overlay.yml

c. Patching ‘ootb-config-writer’
Update ‘source-overlay.yaml’ to include your custom CA and save to file
 

#filename: ootb-config-writer.yaml


#@ load("@ytt:overlay", "overlay")

apiVersion: v1
kind: Secret
metadata:
  name: patch-ootb-templates
  namespace: tap-install
stringData:
  patch.yaml: |
    #@ load("@ytt:overlay", "overlay")

    #@ def inject_ca_cert(left, right):
    #@ return left.replace("set -o xtrace", "set -o xtrace\n\ncat >/etc/ssl/certs/custom-ca.crt <<EOF\n" + right + "\nEOF")
    #@ end

    #@overlay/match by=overlay.subset({"kind":"ClusterTask","metadata":{"name":"image-writer"}})
    ---
    spec:
      steps:
      #@overlay/match by=overlay.all
      -
        #@overlay/replace via=inject_ca_cert
        script: |
                -----BEGIN CERTIFICATE-----
                MIIDgDCCAmigAwpRVUraH2lL…
                -----END CERTIFICATE-----

#@overlay/match by=overlay.subset({"kind":"PackageInstall","metadata":{"name":"ootb-templates"}})
---
metadata:
  #@overlay/match missing_ok=True
  annotations:
    #@overlay/match missing_ok=True
    ext.packaging.carvel.dev/ytt-paths-from-secret-name.0: patch-ootb-templates

Create K8s Secret to store content

kubectl create secret generic -n tap-install tap-patch-ootb-templates-ca --from-file=./ootb-config-writer.yaml

d. Patch TAP "PackageInstall" to load all YTT merge overlays
 

kubectl patch -n tap-install --type merge pkgi tap --patch ‘{“metadata”:{“annotations”:\
{“ext.packaging.carvel.dev/ytt-paths-from-secret-name.0”: “convention-service-cert”, “ext.packaging.carvel.dev/ytt-paths-from-secret-name.1”:“tap-patch-ootb-templates-ca”, “ext.packaging.carvel.dev/ytt-paths-from-secret-name.2”: “source-service-cert”}}}’