[VMware Cloud on AWS] (HCX) - "Adding Mobility Agent Host failed. Cannot complete login due to an incorrect username or password"

Products

• VMware HCX
• VMware Cloud on AWS

Issue/Introduction

To identify how to workaround thtis error message

Symptoms:
HCX Environments running R134 or earler.

Error message symtoms similar to in the "Tasks" tab of HCX service mesh:

[HCX-IX-R1] Configuring Mobility Agent host
[HCX-IX-I1] Adding Mobility  Agent Host failed. Cannot complete login due to an incorrect user name or password
[HCX-WO-I1] Logging in to WAN-OPT explicitly
[HCX-WO-R1] Post Configuring WAN-OPT appliance

or

[Deploy and Configuration of Interconnect Appliances Failed. Interconnect Service Workflow interconnectConfigureMA failed Error ["Adding Mobility Agent Host failed. Cannot complete login due to an incorrect user name or password."]
[HCX-IX-I1] - Adding Mobility Agent Host failed. Cannot complete login due to an incorrect user name or password.

Error Messages in vCenter "Recent Tasks"

Cannot complete login due to an incorrect user name or password.

Cause

The cause is actually likely related to a network configuration problem when you check the HCX logs
/common/logs/admin/app.log

2020-02-25 21:44:12.291 UTC [InterconnectService_SvcThread-11, IX:823808a3-####-####-####-##########08, J:fb97eaeb, , TxId: ae31b326-54c6-446d-afbe-6097f548d2f0] INFO  c.v.v.h.s.i.AppliancePostConfigJob- WAN-OPT login failed. Requeuing back. Re-try attempt 2
2020-02-25 21:44:16.246 UTC [VsphereReplicationService_SvcThread-329, Ent: HybridityAdmin, , TxId: f2cdb8dc-####-####-####-##########45] INFO  RCM- Cleaning the cache since 5
2020-02-25 21:45:16.259 UTC [VsphereReplicationService_SvcThread-341, Ent: HybridityAdmin, , TxId: f2cdb8dc-####-####-####-##########45] INFO  RCM- Cleaning the cache since 5
2020-02-25 21:45:17.375 UTC [InterconnectService_SvcThread-20, IX:823808a3-####-####-####-##########08, J:fb97eaeb, , TxId: ae31b326-####-####-####-##########f0] ERROR c.v.v.h.a.gateway.GatewayAdapter- Exception while posting configuration to gateway.
org.apache.http.conn.ConnectTimeoutException: Connect to 10.2.42.11:9443 [/10.2.42.11] failed: connect timed out
        at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:143)

Resolution

This is related to best practices of how to overcome the error message seen above

Validate and check the following

• End user is able to log into HCX Enterprise Manager :9443 page via admin
• Validate proper administrator privleges are applied to associated vCenter for HCX

Login to https://<on-premises.hcx.mgr.ip>:9443/ --> Select "Configuration" tab --> Select "vCenter" tab and ensure the user has administrative privleges

• Ensure the user is able to log into vCenter and access the HCX plug-in in the vCenter UI (HTML5 client)

Once the above has been checked, then it will be best to check the network profile settings and ensure you have connectivity to your on-premises hosts from the HCX uplink network profile.

This is captured in our public documentation - 
https://docs.vmware.com/en/VMware-HCX/services/user-guide/GUID-184FCA54-D0CB-4931-B0E8-A81CD6120C52.html?hWord=N4IghgNiBcIHIFMAuB3A9gJwNYAIAKGaAZgJYQIgC+QA

Then check:

• That HCX manager is able to communicate with the Service Mesh Management IP (found in app.log)
• Verify no wrong IP range in network profile
• Verify no wrong Gateway in network profile
• Verify no wrong subnet mask in network profile
• Verify no wrong port group in network profile
• Verify no firewall or routing problem between HCX Manager and the service mesh network (uplink)