SDDC Manager accounts disconnected after vIDM CSP-102092 patch

Products

• VMware SDDC Manager
• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• vIDM root account shows disconnected in SDDC manger UI post patching to CSP-102092

• In Password Management Page we see the error:

• We are able to connect to vIDM through SSH using same root password.

• From operationmanager log we could see SSH connections failing with below errors : /var/log/vmware/vcf/operationsmanager/operationmanager.log:

YYYY-MM-DDT23:33:56.313+0000 ERROR [vcf_om, 690152e4d9d713695b3cd981a64ee63f, c2cc] [c.v.e.s.c.u.c.SshCommandExecuter, om-exec-11] Could not connect to the SSH server @ ####.example.com for configuration.
com. vmware. evo.sddc.common.util.command.CommandExecuterException: SSH: Failed to establish SSH session to ####.example.com.
YYYY-MM-DDT23:33:56.314+0000 ERROR [vcf_om,690152e4d9d713695b3cd981a64ee63f,c2cc] [c.v.v.p.u.c.SshPasswordChanger, om-exec-11] Failed to execute command, error : SSH: Failed to establish SSH session to  ####.example.com.
YYYY-MM-DDT23:33:56.314+0000 ERROR [vcf_om, 690152e4d9d713695b3cd981a64ee63f, c2cc] [c.v.v.p.u.c. SshPasswordChanger, om-exec-11] Unable to login to ####.example.com. org with username root got from CSS
[vcf_om,690152e4d9d713695b3cd981a64ee63f,c2cc] [c.v.v.p.u.c.AbstractPasswordChanger, om-exec-11] Failed to execute command, error : SSH: Failed to establish SSH session to  ####.example.com.

Environment

SDDC Manager 5.2.2 with vIDM 3.3.7.0

VCF 9.x

VCF 5.2.2.0

Cause

This problem happens due to the sshd_config on the SDDC Manager appliance disabling RSA algorithms, which the SDDC Manager legacy JSch library requires for automated connections.

Resolution

Fixed in VCF 5.2.3 and higher. See Download Broadcom products and software for steps to download this release.

 

Stronger algorithms are used, and host keys must be re-trusted:

1. Reconfigure SSH on vIDM Nodes

• • Edit /etc/ssh/sshd_config so it reads as follows for our specific configuration lines:

#HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
HostkeyAlgorithms -ssh-rsa

• • Restart SSH:

systemctl restart sshd

2. Re-Trust Host Keys in SDDC Manager

• • Follow KB 316028 and download the fixHostKeys.py script. Store it on the SDDC Manager and run (after you take a SDDC VM snapshot):

3. Remediate Password in SDDC Manager.

• • System administrators must navigate to Administration > Security > Password Management, click the vertical ellipsis next to the service account, and select Remediate

Additional Information

Managing Passwords in VMware Cloud Foundation