health-check certificate expired warning

Products

• Operations Manager

Issue/Introduction

Customer has followed the cert rotation document from Pivotal:
https://techdocs.broadcom.com/us/en/vmware-tanzu/platform/tanzu-operations-manager/3-0/tanzu-ops-manager/security-pcf-infrastructure-rotate-cas-and-leaf-certs.html

A new cert has been generated but the old expired certificate kept showing up even after the new certificate has been generated. 

In order to retrieve certificate from ops manager, you need to do the following to get the access token:

• uaac target https://FQDN/uaa --skip-ssl-validation
• uaac token owner get opsman admin -s "" -p PASSWORD
• uaac context
• save access_token output as value of TOKEN

curl https://opsmgr.my.domain/api/v0/deployed/certificates -H "Authorization: bearer $TOKEN"  |  jq -r '.certificates[] | select( .location == "credhub")'

After generating a new token, customer went back and checked the expiration date of the certificates and still see the certificates that will be expiring soon.

The following is the snip of those expired certificate:

{
  "is_ca": false,

  "property_reference": null,

  "property_type": null,

  "product_guid": "bosh-health-check",

  "configurable": true,

  "issuer": null,

  "valid_from": null,

  "valid_until": "2019-10-04T16:50:12Z",

  "location": "credhub",

  "variable_path": "/dns_api_client_tls"

}

Environment

Product Version: 3.0

Resolution

The cert with the name bosh-health-check is from an old deployment of the Healthcheck tile. It's possible that when the tile was deleted there was an active Healthcheck deployment running which got the old expired certificate left behind. The old deployment is still holding a reference to this old certificate that still shows up in the api call made to display certificates that will soon expire.

run  'bosh deployments' to see if there is a left-over deployment named bosh-health-check. 

Delete manually using the 'bosh delete-deployment' command with the old bosh health-check deployment.