SDDC Manager password remediation for NSX-T Edge nodes failing with Error: Unable to get transport nodes from NSX-T cluster

Products

• VMware Cloud Foundation

Issue/Introduction

To perform list of troubleshooting to resolve the error with password remediation and allow successful credential remediate operation in Password Management tab of the SDDC Manager UI.

Symptoms:
Password remediate for NSX-T Edge node failing with below error in SDDC UI: 

Unable to obtain SSH connectivity to entity: <nsx-t edge node fqdn>



SDDC Manager > operationsmanager.log 



 

 

Environment

Vmware Cloud Foundation 4.5
VMware Cloud Foundation 4.x

Cause

Mismatched hostname of Edge Node in SDDC Manager Platform DB

Resolution

1. ssh to sddc manager with vcf user and su to root
2. Run the below command on SDDC Manager to fetch all the edge nodes deployed 

   • curl -X GET -k -s -w "\n" -u 'admin:<password>' https://<nsxt-manager-fqdn>/api/v1/transport-nodes | json_pp

   • Note: Replace the nsxt-manager fqdn and the password in the curl command
3. From the response of above curl command, pick the id of the edge node
4. Run the below command to list the details of the edge node for which password remediate is failing

   • curl -X GET -k -s -w "\n" -u 'admin:<password>' https://<nsxt-manager-fqdn>/api/v1/transport-nodes/<id> | json_pp

   • Note: Replace the <id> with the edge node id gathered in Step 3.
5. From the above response check the hostname value under node_deployment_info -> node_settings -> hostname
6. The hostname here should exactly match the hostname on SDDC DB ---> Match FQDN / Short name / Lower or upper case.
7. Update nsxt_edge_nodes in nsxt_edge_cluster table with the matching hostname in SDDC Platform DB - Take snapshot of SDDC Manager before making any changes to SDDC DB

   • psql -h localhost -U postgres -d platform
     
     select id,status,name,nsxt_edge_nodes,source_id from from nsxt_edge_cluster;      -------> Note the id of the nsxt_edge_cluster which needs to be updated
     
     update nsxt_edge_cluster set nsxt_edge_nodes='xxxxxx' where id='<nsxt_edge_cluster id noted from above step>';

8. Retry password remediation for NSX-T Edge nodes