Best Practice: Replace a Faulty NSX-T Manager Node in a 3-Node Cluster
search cancel

Best Practice: Replace a Faulty NSX-T Manager Node in a 3-Node Cluster

book

Article ID: 445340

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • One NSX-T Manager node in a 3-node cluster is unrecoverable or corrupt.

  • Cluster status shows as 'Degraded'.

  • This article provides the best practice procedure to delete a faulty NSX-T Manager node, redeploy a new appliance, and join it back to the existing cluster.

Environment

VMware NSX

Resolution

1. Prerequisites

Before beginning the replacement, ensure the following are available:

  • Backups: A recent successful backup of the NSX-T Manager and the backup passphrase.
  • Compute Resources: Verify that the target ESXi host or Resource Pool has sufficient CPU and RAM overhead to support the new NSX Manager node.
  • Networking Details: IP address, netmask, gateway, DNS, and NTP server addresses for the node being replaced. 
  • Credentials: Passwords for admin, audit, and root users.
  • UUID: Collect the UUID of the faulty manager node to ensure no stale associations remain after deletion.
  • Software: Download the exact matching OVA build from the Broadcom Support Portal (e.g., nsx-unified-appliance-4.2.1.0.0.24302016.ova).
  • Node Sizing: Determine the existing node size (Small, Medium, or Large) by checking the hardware of an operational node in vCenter.
  • Power-Off and delete the faulty NSX manager.

2. Procedure

Step 1: Deploy the New NSX-T Manager Appliance

  1. Log into the vSphere Web Client.
  2. Navigate to Hosts and Clusters > [Your-Cluster] > [Mgmt-ResourcePool].
  3. Right-click and select Deploy OVF Template...
  4. Select the downloaded OVA file and follow the wizard to input the networking and credential details gathered in the prerequisites.
  5. Power on the VM and wait for all services to start completely.

Step 2: Collect Cluster Information

  1. SSH into an operational NSX-T Manager node (e.g., Node 1):
  2. ssh admin@<operational_mgr_fqdn_or_ip>
  3. Retrieve the Cluster ID
  4. get cluster config | find Id:
  5. Retrieve the API Thumbprint:
  6. get certificate api thumbprint

Step 3: Join the New Node to the Cluster

  1. SSH into the newly deployed NSX-T Manager node
  2. Run get cluster status on the newly deployed NSX-T manager node and confirm if all the services have come up before joining to the cluster.
  3. Execute the join command using the information retrieved in Step 2:
  4. join <operational_mgr_ip> cluster-id <uuid> thumbprint <thumbprint> username admin
  5. Example: join 198.xx.xx.xx cluster-id 3ca96913-xxxx-xxxx-xxxx-xxxxxxxx thumbprint xxxx...xxxx username admin

3. Post-Checks

  • Cluster Health: Run get cluster status on any node or check the NSX UI to ensure the cluster is stable.
  • Anti-Affinity Rules: Re-verify that the VM anti-affinity rules in vSphere are correctly applied to the new node.
  • UUID Cleanup: Search for the UUID of the deleted node to ensure no stale associations (such as old certificates) remain.

 

Powered by Wolken Software