Deleting Orphaned NSX Certificates and Load Balancer Objects after TKGi Decommissioning fails with error code 3022

Products

• VMware NSX

Issue/Introduction

An alarm indicates an expired or expiring SSL certificate in the NSX Trust Store.

Attempting to delete the certificate via the NSX Manager UI fails with: One or more relationships exist for object with id Certificate/<ID> (Error code: 3022).

The "Used By" column may show a reference to a LbVirtualServer that was associated with a now-decommissioned Tanzu Kubernetes Grid Integrated (TKGi) cluster.

Standard remediation tools like the Certificate Analyzer Resolver (CARR) script fail to release the certificate because it is managed by the NSX Container Plugin (NCP).

Environment

VMware NSX

Cause

When a TKGi cluster is decommissioned or a namespace is deleted, the NSX Container Plugin (NCP) may occasionally fail to clean up all Load Balancer resources. This leaves "orphaned" or "ghost" Virtual Servers in the NSX database that maintain a lock on the associated SSL certificates.

Resolution

Since the TKGi master VMs are no longer available after decommissioning, the cleanup must be performed directly via the NSX Manager API.

Step 1: Identify the Blocking Virtual Server

1. Log into the NSX Manager CLI as admin.
2. Use the following API call to find which Virtual Server is referencing the certificate ID:

   curl -k -u admin -X GET "https://localhost/api/v1/loadbalancer/virtual-servers"

3. Search the output for the Certificate UUID to find the associated virtual_server_id.

Step 2: Force Delete the Virtual Server

To break the relationship, you must delete the orphaned Virtual Server. Because these are NCP-managed objects, a standard delete may fail. Use the X-Allow-Overwrite header to force the operation.

IMP: Validate the VS is not longer in production and take standard NSX backup

curl -k -u admin -X DELETE -H "X-Allow-Overwrite: true" "https://localhost/api/v1/loadbalancer/virtual-servers/<virtual-server-id>"

Step 3: Delete the Expired Certificate

Once the Virtual Server is removed, the certificate's "Used By" count should drop to 0, allowing for deletion via the UI or API:

 

curl -k -u admin -X DELETE "https://localhost/api/v1/trust-management/certificates/<certificate-id>"