VCF Upgrade fails with message " MGT domain failed upgrade"

Products

• VMware Cloud Foundation

Issue/Introduction

To resolve the SSH authentication 

VCF Upgrade fails with message "MGT domain failed upgrade. Please resolve the above upgrade failure for this bundle before applying any other available bundle"

lcm logs similar to below

DEBUG [,,,] 7233 --- [pool-4-thread-7] com.vmware.vcf.secure.ssh.SshExecuter : Establishing SSH session to host: <FQDN-ESXiHost>
DEBUG [,,,] 7233 --- [pool-4-thread-7] v.s.c.s.SecurityConfigurationServiceImpl : Security config retrieved class SecurityConfig {
certificateValidationEnabled: true
fipsMode: false
}
ERROR [,,,] 7233 --- [pool-4-thread-7] com.vmware.evo.sddc.common.util.SshUtil : Unable to create jsch CLI session: com.jcraft.jsch.JSchException: java.net.ConnectException: Connection refused (Connection refused) at com.jcraft.jsch.Util.createSocket(Util.java:394) ~[jsch-0.1.55.jar:na]
ERROR [,,,] 7233 --- [pool-4-thread-7] c.v.e.s.o.model.error.ErrorFactory : [CVASCN] FAILED_TO_UPDATE_LSOM_SETTINGS_ON_HOST Failed to update vSAN LSOM settings on the host <FQDN-ESXiHost>
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to update vSAN LSOM settings on the host <FQDN-ESXiHost> at com.vmware.vcf.migration.actions.UpdateLSOMValuesAction.updateLsomSettings(UpdateLSOMValuesAction.java:87) ~[sddcmanager-migration-app-4.3.1-vcf4310RELEASE.jar:na]

Environment

Vmware Cloud Foundation 4.5.1
Vmware Cloud Foundation 4.5
VMware Cloud Foundation 5.0

Cause

This occurs due to SSH service  not running on ESXi hosts, and hence SSH connection is refused and  config drift upgrade fails.

Resolution

To resolve this issue enable the SSH service on these ESXi  hosts and re-trigger config drift update.

Steps to enable SSH on each host:
1. On vSphere client, navigate to the host(under vCenter -> Datacenter-> cluster)
2. Navigate to Configure tab -> [System] dropdown -> Services -> Select "SSH" from the services list -> click on [start] after [SSH] service is highlighted.
3. Repeat 1,2 for all the hosts which needs SSH service to be enabled

Additional Information

Impact/Risks:
None