VCF Installer Fails SDDC Specification Validation with error "SSH: Failed to establish SSH session to <VCF Installer FQDN> Auth fail" due to 'DNS Domain Name' filled with FQDN instead of Domain Name

Products

• VMware SDDC Manager / VCF Installer
• VMware Cloud Foundation

Issue/Introduction

• Validating the SDDC specification from the VCF Installer (VCF 9.x, Brownfield conversion) failed with the error: "SSH: Failed to establish SSH session to <VCF Installer FQDN>  Auth fail".

• /var/log/vmware/vcf/domainmanager/domainmanager.log on VCF Installer shows below error,

YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,#############] [c.v.e.s.c.u.c.SshCommandExecuter,http-nio-127.0.0.1-7200-exec-2] Could not connect to the SSH server @ <VCF Installer FQDN> for configuration.
YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,#############] [c.v.e.s.v.util.DnsAndNtpUtil,http-nio-127.0.0.1-7200-exec-2] Failed to obtain DNS information from VCF Installer with hostname<VCF Installer FQDN>
YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,#############] [c.v.v.v.c.v1.VcfInstallerController,http-nio-127.0.0.1-7200-exec-2] Failed to validate SDDC specification.
...
YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,#############] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7200-exec-2] [18P48G] FAILED_TO_GET_DNS_INFO Failed to get DNS information
YYYY-MM-DDTHH:MM:SS DEBUG [vcf_dm,#############] [c.v.e.s.e.h.LocalizableRuntimeExceptionHandler,http-nio-127.0.0.1-7200-exec-2] Handler Error Response: {"errorCode":"FAILED_TO_GET_DNS_INFO","arguments":[],"message":"Failed to get DNS information","causes":[{"type":"com.vmware.evo.sddc.common.util.command.CommandExecuterException","message":"SSH: Failed to establish SSH session to <VCF Installer FQDN>"},{"type":"com.jcraft.jsch.JSchException","message":"Auth fail"}],"referenceToken":"18P48G"}

Environment

 VCF 9.x

Cause

Issue caused by incorrect DNS domain name configuration within the VCF Installer. During VCF Installer OVA deployment, 'DNS Domain Name' field updated with Fully Qualified Domain Name (FQDN) of an VCF Installer appliance (e.g., <VCF Installer FQDN>) instead of the correct base domain name (e.g.,<DNS Domain Name>).

Resolution

To resolve this issue,

1. Power off the current VCF Installer VM.
   
   
2. Deploy new VCF Installer again with OVA file. (Same IP address can be used as the old VCF installer is powered off)
   
   
3. On "Deploy OVF Template" wizard, under "Customize template", ensure 'DNS Domain Name' field is with domain name as value (e.g.,<DNS Domain Name>) and NOT the FQDN (e.g., <VCF Installer FQDN>).
   Note: 'Host Name' field should be filled with FQDN.
   
   
4. Power on the VCF Installer VM and go through Deployment Wizard.
   Note: Alternate option to avoid going through complete Deployment Wizard process is by download the JSON spec file from Deployment Wizard before powering off the old VCF installer VM and use the same JSON spec file in the new VCF Installer using option "Deploy using JSON Spec". (Warning: The JSON spec saves all credentials in plain text, ensure to store the file at a secure location.)

Additional Information

If you see below error on /var/log/vmware/vcf/domainmanager/domainmanager.log, please follow KB article 441496 for resolution

YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm#############] [c.v.e.s.v.v.SpecComponentValidator,http-nio-127.0.0.1-7200-exec-1]  Invalid local user password specified in SDDC Manager specification