Why /services/tls_ca rotation triggers gorouter and diego_cell update

Products

• Operations Manager

Issue/Introduction

Services TLS CA (/services/tls_ca) is used to sign service relevant leaf certificates, it is excluded by Operations Manager certificate rotation API, thus has to be rotated with Credhub maestro tool. During the rotation, TAS and Isolation Segment router and diego_cell instances are triggered to be updated, which takes significant time for large scale of platforms.

Cause

Services TLS CA (/services/tls_ca) is not only used by Tanzu Platform service tiles, but also by router and diego_cell instance groups, which rely on CA to access service instances via TLS. The detailed connection can be confirmed at https://<OPSMAN>/debug/files. 

Router instance group includes Services TLS CA in gorouter config file with Diego Instance Identify Intermediate CA and other CAs. 

  - "((/services/tls_leaf.ca))"
  - "((diego-instance-identity-intermediate-ca-2-7.ca))"
  - "((/opsmgr/cf-***/network_policy_server_external_cert.cert_pem))"

Diego_cell instance group includes it under cflinuxfs properties, so that apps on diego_cell can establish TLS connection with the backend service instances without additional manual settings. 

    properties:
      cflinuxfs3-rootfs:
        trusted_certs: |+
          ((/services/tls_leaf.ca))
          ((diego-instance-identity-intermediate-ca-2-7.ca))
    ...

    properties:
      cflinuxfs4-rootfs:
        trusted_certs: |+
          ((/services/tls_leaf.ca))
          ((diego-instance-identity-intermediate-ca-2-7.ca))

Resolution

When rotating Services TLS CA (/services/tls_ca) on a platform with massive router and diego_cell instances, it is important to prepare sufficient maintenance time window because of time for updating those instances.