Certificate not updated after updating avi-controller-ca secret

Products

• Tanzu Kubernetes Runtime

Issue/Introduction

• AVI Controller Certificate not updated after updating avi-controller-ca secret in TKG
  
  
• The AVI Controller CA has been updated by patching the avi-controller-ca as per the docs on how to modify the Avi Controller Certificates
  
  
• The ako and ako-operator show that they can't connect to the AVI Controller due to an unknown certificate authority.
  avisession.go:666] Client error for URI: login. Error: Post "https://<AVI Controller>/login": tls: failed to verify certificate: x509: certificate signed by unknown authority

Environment

2.5.4

Cause

During AVI CA rotation, the new CA was patched into the cluster secrets, but AKO continued using a stale or incomplete trust bundle due to secret name/namespace mismatch 

Resolution

1. Update the tkg-pkg-system-values with the new certificate.
   kubectl get secret -n tkg-system tkg-pkg-system-values -o jsonpath='{.data.tkgpackagevalues\.yaml}' | base64 -d > tkgpackagevalues.yaml
   
   Note: In TKG 2.4 releases, the secret name is changed to tkg-pkg-tkg-system-addon in the tkg-system namespace. 
   
   
2. Update CA in tkgpackagevalues.yaml
   
   
3. Encode the updated tkgpackagevalues.yaml
   cat tkgpackagevalues.yaml | base64 -w 0
   
   
4. Update  tkg-pkg-tkg-system-addon secret with encoded contents of tkgpackagevalues.yaml
   kubectl  edit secret -n tkg-system tkg-pkg-tkg-system-addon