How to delete orphan NSX-T objects protected by superuser
search cancel

How to delete orphan NSX-T objects protected by superuser

book

Article ID: 298537

calendar_today

Updated On:

Products

VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

In PKS 1.1.x when creating a cluster using the PKS CLI, the NSX-T container plugin (NCP) creates the network objects in NSX-T which are required for the cluster to function properly.

Cluster deletion through the PKS CLI takes care of cleaning up these objects in NSX-T. However, when a cluster creation attempt fails there are certain orphan objects left behind in the NSX-T environment which requires manual cleanup. These objects can be identified using the failed cluster's UUID and searching it in the NSX-T manager UI. Prior to v1.1.x, these objects could be simply deleted from the UI but that is no longer the case.

The procedure described in this article details how to delete those objects.

Environment


Cause

PKS 1.1.x introduced a new security feature, NSX Manager Super User Principal Identity Certificate, which makes cluster deletion trickier.

Any objects created by this super user can not be deleted through the UI. The only way to delete these objects is by using NSX-T API.

If there are large number of objects left over, deleting them can take a lot of time. A faster way is to:

1. Delete the principal identity created during PKS installation.

2. Delete the orphan objects using NSX manager UI. 

3. Add the principal identity back.

Resolution

Prepare environment variables

Use the same principal identity name, certificate and key generated while configuring PKS using these steps.

export NSX_MANAGER_USERNAME="admin" 
export NSX_MANAGER_PASSWORD="######"
export NSX_MANAGER_IP="oom.domain.com"
export PI_NAME="<Name of principal identity>" 
export NSX_SUPERUSER_CERT_FILE="<Name of principal identity cert file>"
export NSX_SUPERUSER_KEY_FILE="<Name of principal identity cert key>"

Get the principal identity details

There could be two types of principal identity that exists for the cluster:

1. A principal identity is created using the these steps. This needs to be deleted and added back.

2. A principal identity created automatically with the failed PKS cluster's UUID in it's name. If it exists, it has to be deleted. There is no need to add this type of principal identity back as the cluster creation has already failed. 

Use one of the following methods to get principal identity details:

  • Note the Principal Identity ID, Node ID and Certificate ID from NSX Manager -> System -> Users > Principal Identities. This information will be used in later steps.
  • Using the curl command:
    curl -k -X GET "https://${NSX_MANAGER_IP}/api/v1/trust-management/principal-identities" -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD"

Delete the principal identity

The below curl command can be used to delete the principal identity. Use the Principal Identity ID identified in the steps above.

curl -k -X DELETE \
"https://${NSX_MANAGER_IP}/api/v1/trust-management/principal-identities/<Insert Principal Identity ID here>" \
-u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD"

Delete orphan NSX-T objects 

Now the orphan NSX-T objects can be deleted from the UI. To identify the objects, determine the failed cluster's UUID using PKS clusters. Paste this ID in the NSX-T manager's search bar to get a list of objects.


Add Principal Identity 

Please remember to add back the principal identity as this is mandatory for cluster creation and all the NCP operations. To add back the principal identity, the following environment variables are required:

export PI_NAME="<Name of principal identity>"
export NSX_SUPERUSER_CERT_FILE="<Name of principal identity cert file>"
export NSX_SUPERUSER_KEY_FILE="<Name of principal identity cert key>"
export CERTIFICATE_ID="<Certificate ID identified earlier >"
export NODE_ID="<Node ID identified earlier>"

pi_request=$(cat <<END
{
"display_name": "$PI_NAME",
"name": "$PI_NAME",
"permission_group": "superusers",
"certificate_id": "$CERTIFICATE_ID",
"node_id": "$NODE_ID"
}
END
)

curl -k -X POST \
  "https://${NSX_MANAGER_IP}/api/v1/trust-management/principal-identities" \
  -u "$NSX_MANAGER_USERNAME:$NSX_MANAGER_PASSWORD" \
  -H 'content-type: application/json' \
  -d "$pi_request"