SDDC Manager fails to find NSX Manager nodes due to case-sensitivity in FQDN
Article ID: 450336
Updated On:
Products
Issue/Introduction
Within VMware Cloud Foundation (VCF), lifecycle management or orchestration tasks involving NSX-T fail. The following symptoms are observed:
• SDDC Manager UI: The task fails with error code 9COAQD and message: FAILED_TO_FIND_THE_NSX_VMS Failed to find the virtual machines for the NSX Manager nodes [...].
• Logs: The /var/log/vmware/vcf/operationsmanager/operationsmanager.log contains an error from DiscoverNsxtClusterUtil indicating an empty mapping: Could not find VM names for the NSXT cluster node VMs. Actual NSXT cluster nodes: [vmwarensx01.example.com], NSXT cluster node FQDN to VM name mappings: {}
Environment
VMware Cloud Foundation 9.x
Cause
This issue occurs due to a case-sensitivity mismatch between the SDDC Manager platform database and the vCenter Server inventory.
SDDC Manager orchestration logic requires an exact string match to map FQDNs to VM objects. If the nsxt_cluster_details in the database contains uppercase characters (e.g., VMWARENSX01.example.com) while the vCenter inventory uses lowercase, the mapping fails, and the VMs are not discovered.
Resolution
To resolve this issue, you must manually update the SDDC Manager database to ensure the FQDNs in the inventory records match the casing of the actual virtual machines in vCenter.
⚠️ IMPORTANT: Always take a snapshot of the SDDC Manager appliance before performing manual database modifications.
1. Identify the affected NSX ID: SSH to the SDDC Manager as vcf and elevate to root.
Connect to the platform database:
psql -h localhost -U postgres -d platform
Run the following to find the current configuration:
SELECT id, nsxt_cluster_details FROM nsxt;
2. Update the Database Record: Update the FQDNs in the nsxt_cluster_details JSON array to lowercase. Ensure the vmName, id, and ipAddress fields remain unchanged from the original output.
⚠️ IMPORTANT: Verify every parameter before running this command. This operation cannot be undone.
sql UPDATE nsxt SET nsxt_cluster_details ='[{"fqdn":"nsx01.example.com","vmName":"...","ipAddress":"...","id":"..."}, ...]' WHERE id ='<UUID>';
3. Restart SDDC Manager Services: Exit the database (\q) and run the restart script: /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh
4. Verification: Retry the failed task in the SDDC Manager UI. The orchestration should now successfully map the FQDNs to the VMs and proceed.
Additional Information
Feedback
