Unable to delete stale attachments of node associated with expired pks certificates

Products

• VMware NSX

Issue/Introduction

• PKS (Pivotal Kubernetes Service) Principle Identity (PI) certificates expired and still have nodes attached, which generate alarm:

Note: PKS PI Certificate, refers to a certificate used by PKS to securely authenticate and manage NSX-T resources through its API, acting as a superuser.

• In the NSX manager, System, Certificates, there are expired PKS certificates which have still nodes attached:

• When you click on 'where used' on that certificate, we can see its attached to a NSX manager node ID with 'Client Auth' service: (For the same NSX manager node ID there is already a valid certificate with 'Client Auth' service):

Environment

VMware NSX

Cause

These expired PKS PI certificates, have 'Client Auth' Service attached to a NSX manager node, the same NSX manager node has a valid (not expired) PKS certificate with the 'Client Auth' service.

This issue can occur when the certificate was replaced, as it could not detach the Client Auth Service from the expired certificate at the time of replacement and as it is still attached to a manager node, we unable to delete the certificate.

Resolution

If you encounter this issue, run the CARR script attached to this KB: Using Certificate Analyzer, Results and Recovery (CARR) Script to fix certificate related issues in NSX

If the issue persists after running the CARR script, please open a support request with Broadcom NSX support and reference this KB.

Additional Information

If you are opening a support request, please provide the carr.log after running the script.

A screenshot of the issue.

NSX manager logs.

Results of the following API call: GET https://<nsx-manager-ip>/api/v1/trust-management/certificates