Due to network disconnection, Password remediation for the NSX Admin account on the SDDC Manager fails with the error "No route to host"

Products

• VMware SDDC Manager

Issue/Introduction

• Admin and/or audit account(s) for NSX manager are found in disconnected state under Password Management on SDDC manager:

 

• The password for the NSX admin user account has been changed outside of SDDC Manager following the steps outlined in the KB- NSX Edge Nodes Disconnected in Password Manager on SDDC and one can successfully authenticate to the NSX Manager node with that credential but attempting to remediate it on the SDDC Manager fails with below error:

I/O error on GET request for "https://<nsx_manager_fqdn>/api/v1/node/users": No route to host

• Upon reviewing the SDDC manager logs with respect to the password remediation task, below entries are observed pointing to network issue:

less /var/log/vmware/vcf/operationsmanager/operationsmanager.log

YYYY-MM-DDThh:mm:ss DEBUG [vcf_om, 68f67f9ea68293d142d92429b4f#####, ed##] [c.v.v.p.u.c.AbstractPasswordChanger, om-exec-8] Error Message : I/O error on GET request for "https://<nsx_manager_fqdn>/api/vl/node/users": No route to host, Error Token :<Error_Token_id>, Error Cause : {}

• Attempting to ping the NSX manager IP and FQDN from the SDDC manager fails with "Host unreachable".

Environment

VCF 5.x

Cause

As mentioned on the log snippet and further verified from the ping results, there is network connectivity issue between the NSX manager and the SDDC manager . As a result SDDC manager is failing to reach the NSX manager with the credentials and the password remediation attempt fails. 

Resolution

Test connectivity from SDDC manager to the NSX. SSH into the SDDC Manager and run the following command:

ping <nsx_manager_fqdn_or_ip>

Below is the output of a successful ping:

If it times out, engage your Networking team to investigate and restore the connectivity between NSX manager and SDDC manager.

In case you need further support kindly engage Broadcom support.

Additional Information

NSX Edge Nodes Disconnected in Password Manager on SDDC