The Quotas tab and Profile tab on the TKGi Management Console showing x509: certificate signed by unknown authority.

Products

• VMware Tanzu Kubernetes Grid Integrated (TKGi)

Issue/Introduction

TanzuSymptoms:

• The Quotas tab and Profile tab on the TKGi Management Console showing x509: certificate signed by unknown authority.

Failed to retrieve quotas information. cannot login into TKGI: Post https://<TKGI_API_URL>:8443/oauth/token: x509: certificate signed by unknown authority.

Failed to fetch compute profile list cannot login into TKGI: Post https://<TKGI_API_URL>:8443/oauth/token: x509: certificate signed by unknown authority

• This issue emerges after rotating the configurable Leaf Certificates for the TKGI API from Ops Manager, and not from TKGi Management Console

Environment

VMware Tanzu Kubernetes Grid Integrated Edition 1.x

Cause

Rotating the configurable Leaf Certificates for the TKGI API from opsman side and not from TKGi Management Console side, when the "Manage Certificates Manually for TKGI API" option is not enabled on the  TKGi Management Console TKGiMC >> TKGI Configuration >> Identity.

As there will be a mismatch between the  Certificate to secure the TKGI API on the TKGIMC and the configurable Leaf Certificates TKGI Tile 

Resolution

Note: 

• These steps need to be performe after the  rotating  the configurable Leaf Certificates  for the TKGI API from opsman and not for the TtKGi Management Console and the apply change completed succesfully.
• If the  configurable Leaf Certificates  for the Harbor tile was rotated using the opsman too, one need to apply steps 6 to 8 and then update the harbor section 

 

1. Login to the TKGiMC UI >> Deployment Metadata  and make a note of the "Ops Manager Passphrase"
2. SSH into the Ops Manager VM by following the steps in Log Into the Ops Manager VM with SSH.
3. On the command line, navigate to the scripts directory:

   # cd /home/tempest-web/tempest/web/scripts/
   

4. Run the following command to decrypt the installation YAML file and make a temporary copy of the decrypted file. When prompted for a passphrase, enter the decryption passphrase you created when you launched Ops Manager for the first time:

# sudo -u tempest-web SECRET_KEY_BASE="s" ./decrypt /var/tempest/workspaces/default/installation.yml /tmp/installation.yml

Enter the Ops Manager Passphrase when get promoted. 

Ex:
ubuntu@opsman-local:/home/tempest-web/tempest/web/scripts$ sudo -u tempest-web SECRET_KEY_BASE="s" ./decrypt /var/tempest/workspaces/default/installation.yml /tmp/installation.yml
fatal: Not a git repository (or any of the parent directories): .git
Passphrase:

5. Open /tmp/installation.yml.
6. Find the pivotal-container-service guid certificate and Key under pks_tls and make a note of the "private_key_pem" and "cert_pem"

Ex:
- guid: pivotal-container-service-4ca7cb097af20375fbeb
    installation_name: pivotal-container-service
    vm_type_id: medium.disk
    disk_type_id: '10240'
    properties:
    - deployed: true
      identifier: pks_tls
      value:
        private_key_pem: |-
          -----BEGIN PRIVATE KEY-----
          MIIJRAIBADANBgkqhkiG9w0BAQEFAASCCS4wggkqAgEAAoICAQC+dziVQ/U1LUaS
          1h6i6/TqVKhfB2TjBvy/P7fEHg8wBM9oyBgviJgktkFLX/rejRVPtaQ63I2YHyRO
         ...
          zkrdsBEJ8/YXb9g+lId2Qpaj1MO6lgJTwLUkKTsDZ9hBindmBVAlAnVQzRxKeald
          I2b/u5gfwdfn/3z+JNpdcdc1A4cX7Qdi
          -----END PRIVATE KEY-----
        cert_pem: |-
          -----BEGIN CERTIFICATE-----
          MIIHiDCCBXCgAwIBAgITHQAAAAkDm8eswM8dBgAAAAAACTANBgkqhkiG9w0BAQsF
          ADBIMRUwEwYKCZImiZPyLGQBGRYFdGFuenUxFDASBgoJkiaJk/IsZAEZFgRjb3Jw
         ...          
          LrxsDU58LYfVcwKSuryS5Rv9Kh0tZcFH2zpzQJDgMoZqPqZHFxhiV+w4KAD7WQxd
          R22CcKK+kduUjv0X
          -----END CERTIFICATE-----

7. Confirm that the cert_pem match the "Certificate to secure the TKGI API" on the Tanzu Kubernetes Grid Integrated tile >> TKGi API >> Certificate to secure the TKGI API.
8. Once confireming that the both certficate match, login to the TKGI MC UI .
9. Click on Configuration >> Identity  >> Enable "Manage Certificates Manually for TKGI API" option  by selecting the box infront of it.
10. update the "TKGI API Certificate" field with the TKGi API  "cert_pem"   from  step6 and update the "Private Key PEM" with the "private_key_pem" from step 6 then click next.

Note:

• You will need to remove the 10 spaces from the private_key_pem and cert_pem before updating the corspaning field in TKGI MC using notepad++ .

11. Click on Genrate Configuration >>  Apply Configuration >> Continue

Note:

• The Apply Configuration will create a bosh tasks..

12. Once apply change completed successfully and the TKGI API and DB gets created the Quotas tab and Profile tab on the TKGi Management Console will stop showing x509: certificate signed by unknown authority.