NSX Manager nodes added out-of-band do not sync in SDDC Manager (VCF) 9.0.x

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

NSX Manager nodes deployed directly via the NSX Manager UI (Out-of-Band) are not automatically detected or synchronized by SDDC Manager. This inventory discrepancy prevents management tasks such as CA-signed certificate replacement via Fleet Management and cluster upgrades.

Symptoms:

• SDDC Manager inventory shows only one NSX Manager node despite a three-node cluster being visible in the NSX UI.
• Certificate replacement tasks in Fleet Management fail to identify Node 2 and Node 3.
• NSX CLI command get cluster status shows a healthy three-node cluster.
• SDDC Manager API GET /v1/nsxt-clusters returns only one node spec.

Environment

• VCF Operations 9.0.x
• SDDC Manager
• NSX-T 4.x

Cause

SDDC Manager maintains a managed resource database that is only populated when scale-out operations are orchestrated through the SDDC Manager interface or API. Nodes added OOB lack the necessary UUID and network metadata in the SDDC nsx_manager tables.

Resolution

To resolve the issue, NSX Manager scale out must be performed exclusively via SDDC Manager; Broadcom is currently updating technical documentation to formalize this limitation.

Perform the following steps to reconcile the inventory:

1. Back up SDDC Manager and NSX-T.
2. Log in via SSH to the primary NSX Manager node (the node originally deployed by SDDC Manager).
3. Power off the two OOB secondary nodes in vCenter.
4. Run the command deactivate cluster from the primary node CLI.
   • Reference Link: Deactivate the NSX Manager Cluster
5. Delete the two OOB NSX Manager VMs from the vCenter inventory.
6. Log in to the SDDC Manager UI and navigate to Developer Center > API Explorer.
7. Under NSX-T Cluster, execute GET /v1/nsxt-clusters to retrieve the nsxt-cluster-id.
   
8. Execute a POST /v1/nsxt-clusters/{nsxt-cluster-id}/scale-out using the following JSON payload:

   {
     "nsxManagerSpecs": [
       {
         "name": "####-nsx-node2",
         "networkDetailsSpec": {
           "ipAddress": "####.####.####.####",
           "dnsName": "####-nsx-node2.example.com",
           "gateway": "####.####.####.####",
           "subnetMask": "####.####.####.####"
         }
       },
       {
         "name": "####-nsx-node3",
         "networkDetailsSpec": {
           "ipAddress": "####.####.####.####",
           "dnsName": "####-nsx-node3.example.com",
           "gateway": "####.####.####.####",
           "subnetMask": "####.####.####.####"
         }
       }
     ]
   }

9. Verify the response is 202 Accepted.
10. Monitor the SDDC Manager dashboard until the inventory reflects all three nodes as Healthy.

Post-validation (Full Synchronization may take up to an hour.)

1. SDDC Manager UI
   1. NSX shows as healthy
      • Inventory now reflects 3 NSX Managers
   2. NSX UI
      • System > Appliances: 3 managers, stable cluster

Additional Information

• Refer to API guide: Scale Out Nsx
• For more information on API usage, see the VCF API Guide.
• Subscribe to this article to receive updates on future fixes: How to subscribe to Knowledge Base articles.