Records of detached NSX manager nodes exist after deactivating the cluster
search cancel

Records of detached NSX manager nodes exist after deactivating the cluster

book

Article ID: 322525

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • You've recently deactivated the management cluster in NSX
  • Running the following command on ESXi host shows manager nodes that no longer exist:
> nsxcli -c get controllers
<time-stamp>
 Controller IP Port SSL Status Is Physical Master Session State Controller FQDN
 <removed-nsx-mgr-ip> 1235 enabled not used false null NA
 <esxiting-nsx-mgr-ip> 1235 enabled connected true up NA
 <removed-nsx-mgr-ip> 1235 enabled not used false null NA
  • Result of the follow API call shows controllers that no longer exist in the environment:
GET /api/v1/cluster/nodes
...
{
  "controller_role" : {
    "control_plane_listen_addr" : {
      "ip_address" : "<removed-nsx-mgr-ip>",
      "port" : 1235,
      "certificate" : "-----BEGIN CERTIFICATE-----
...
"resource_type" : "ClusterNodeConfig",
  "id" : "<removed-nx-mgr-uuid>",
...
  "_revision" : 2
}
...
  • If you run the following command on the NSX manager admin CLI you observe only one node:
> get cluster config
<time-stamp>
Cluster Id: <cluster-uuid>
Cluster Configuration Version: 6
Number of nodes in the cluster: 1
Node UUID: <esxiting-nsx-mgr-ip>
Node Status: JOINED
 ...

Environment

VMware NSX-T Data Center 3.2.2

Resolution

This is a known issue impacting VMware NSX-T Data Center

Workaround:
Make sure you have an up to date backup in place before proceeding.
Run the following API calls to remove the records of the controllers that are no longer in use:

  • POST /api/v1/cluster/nodes/deployments/<removed-nsx-mgr-uuid>?action=delete&force_delete=true
  • POST /nsxapi/api/v1/cluster?action=deregister_local_mpa
{"node_uuid": <removed-nsx-mgr-uuid>}
  • DELETE /nsxapi/api/v1/cluster/nodes/<removed-nsx-mgr-uuid>