An expired or expiring client auth certificate is present in the NSX inventory

Products

• VMware NSX
• Carbon Black Cloud Workload

Issue/Introduction

• An alarm indicating a certificate is expiring or expired is generated in the NSX UI
• An expired or expiring certificate with a name starting with CWP is present in the NSX inventory on the System > Settings > Certificates page
• The category of the certificate is client auth and it is a self-signed certificate
• Carbon Black Cloud Workload is installed and integrated with NSX via the Carbon Black Workload (CWP) Appliance
• When you log into the CWP appliance UI, you do not see an NSX details pane. You may also see that the SSO pane has no data
• Re-registering the vCenter Server in the CWP appliance UI brings the NSX details pane back but results in an additional, non-expiring/expired CWP certificate being present in in the NSX UI
• The expired/expiring certificate cannot be deleted (the delete option is greyed out) via the NSX UI

Environment

VMware NSX
VMware NSX-T Data Center
Carbon Black Cloud Workload

Cause

This issue can occur when there is an issue in the CWP appliance affecting the vCenter Server and/or NSX registration

Resolution

After correcting the issue impacting the CWP appliance, if there are two CWP certificates present in the NSX inventory, the expiring/expired certificate can be removed.

As the CWP certificate is associated with a Principal Identity (PI) user, you must first delete the older PI user before the certificate can be deleted.

1. Use the NSX API to list the PI users
   Note: You may see more than two PI users. The only ones of concern for this issue are the CWP PI users
   
   GET https://<NSX Manager IP/FQDN>/api/v1/trust-management/principal-identities
   
   Note: You will see output similar to the following
   
   new PI user and certificate:
   
   

   {
     "results": [
       {
         "name": "CWP",
         "node_id": "c7ba44e2-####-####-####-5be39602ad75_1737817300112",
         "role": "enterprise_admin",
         "certificate_id": "4065d9ac-####-####-####-7fb63cfbbbd5",
         "roles_for_paths": [
           {
             "path": "/",
             "roles": [
               {
                 "role": "enterprise_admin"
               }
             ],
             "delete_path": false
           }
         ],
         "is_protected": true,
         "resource_type": "PrincipalIdentity",
         "id": "b9213ad2-####-####-####-7a6c57a44b4f",
         "display_name": "CWP@c7ba44e2-####-####-####-5be39602ad75_1737817300112",
         "_system_owned": false,
         "_protection": "NOT_PROTECTED",
         "_create_time": 1737817302820,
         "_create_user": "admin",
         "_last_modified_time": 1737817302820,
         "_last_modified_user": "admin",
         "_revision": 0
       },

   
   old PI user and certificate:
   
   

       {
         "name": "CWP",
         "node_id": "c7ba44e2-####-####-####-5be39602ad75_1729868045897",
         "role": "enterprise_admin",
         "certificate_id": "4e500f13-####-####-####-d722af8a6e96",
         "roles_for_paths": [
           {
             "path": "/",
             "roles": [
               {
                 "role": "enterprise_admin"
               }
             ],
             "delete_path": false
           }
         ],
         "is_protected": true,
         "resource_type": "PrincipalIdentity",
         "id": "4ff2b6a0-####-####-####-54f33d9d5144",
         "display_name": "CWP@c7ba44e2-####-####-####-5be39602ad75_1729868045897",
         "_system_owned": false,
         "_protection": "NOT_PROTECTED",
         "_create_time": 1729868048355,
         "_create_user": "admin",
         "_last_modified_time": 1729868048355,
         "_last_modified_user": "admin",
         "_revision": 0
       }
     ]
   }

   
   Note: You can validate which is the old and new PI user by the certificate ID value in the NSX UI. In this example, the older certificate has an ID of 4e500f13-####-####-####-d722af8a6e96, indicating that the associated PI user has an ID of 4ff2b6a0-####-####-####-54f33d9d5144.

2. Remove the older PI user via the NSX API
   
   DELETE https://<NSX Manager IP/FQDN>/api/v1/trust-management/principal-identities/<PI user ID> noted in Step 1
   
   
3. If the "Used By" column in the NSX UI for the expiring or expired CPW certificate now shows "0" after refreshing the Certificates page, click the three vertical dots next to the certificate and select Delete.

Additional Information

This same resolution can be used for removing unneeded PI certificates that are not related to CPW. You would need to identify the associated PI user and ensure that there is no client application making use of the certificate before deleting the PI user and certificate