Health-check operation SDDC fails with DNS-check | Forward DNS lookup is NOT configured as expected

Products

• VMware SDDC Manager

Issue/Introduction

• DNS health check on SDDC fails with error.
• Running /opt/vmware/sddc-support/sos --dns-health also fails with error.
  DNS lookup Status : RED

"name": "sddc01.domain.com: Forward DNS lookup check",
"status": "FAILED",
"message": "Forward DNS lookup is NOT configured as expected, resolved ip IPADDRESS is not equal to IPADDRESS",
"errors": "Forward DNS lookup is NOT configured as expected, resolved ip IPADDRESS is not equal to IPADDRESS"

• vcf-sos.log shows error logging similar to: /var/log/vmware/vcf/sddc-support/vcf-sos.log

INFO [vcf_sos] [progressreporter.py::update_task_status::299::_parallel_check_nsx_dns_configuredThread0] Overall status for DNS-CHECK : COMPLETED_WITH_FAILURE
INFO [vcf_sos] [progressreporter.py::update_task_status::338::_parallel_check_nsx_dns_configuredThread0] Current Status: Task-Name: DNS-CHECK, Entity-Name: edge02.domain.com: Forward DNS lookup check, Task-Status: COMPLETED
INFO [vcf_sos] [progressreporter.py::determine_progress::108::_parallel_check_nsx_dns_configuredThread0] All Tasks : {'VCF-SUMMARY': 'NOT_COMPLETED', 'DNS-CHECK:nsx.domain.com: Reverse DNS lookup check': 'COMPLETED', 'DNS-CHECK:vcsa01.domain.com: Reverse DNS lookup check': 'COMPLETED', 'DNS-CHECK:sddc01.domain.com: Reverse DNS lookup check': 'COMPLETED', 'DNS-CHECK:nsx.domain.com: Forwar
d DNS lookup check': 'COMPLETED', 'DNS-CHECK:esxi03.domain.com: Reverse DNS lookup check': 'COMPLETED', 'DNS-CHECK:esxi02.domain.com: Reverse DNS lookup check': 'COMPLETED', 'DNS-CHECK:esxi04.domain.com: Reverse DNS lookup check': 'COMPLETED', 'DNS-CHECK:esxi01.domain.com: Reverse DNS lookup check': 'COMPLETED', 'DNS-CHECK:sddc01.domain.com: Forward DNS lookup check': 'FAILED', 

Environment

SDDC Manager 5.x

Cause

Issue is caused due to improperly formatted hosts file in the SDDC Manager.

Resolution

To resolve this issue, correct the /etc/hosts file configuration:

1. SSH to the SDDC Manager appliance as the vcf user.
2. Switch to root user:
   
   su -
   
   
3. Create a backup of the current hosts file:
   
   cp -p /etc/hosts /etc/hosts.backup
   
   
4. Check the current /etc/hosts configuration:

cat /etc/hosts
   
   

Workaround 1:

• Navigate to hosts file by using  below command

cat /etc/hosts

# Begin /etc/hosts (network card version)
# End /etc/hosts (network card version)
# VAMI_EDIT_BEGIN
# Generated by Studio VAMI service. Do not modify manually.

127.0.0.1   sddc01 localhost

# VAMI_EDIT_END

• In this case as we can see loopback (127.0.0.1) is not associated with sddc-manager FQDN.
• To fix the issue, edit the hosts file and make sure loopback entry is properly formatted like below:

SDDCIPAddress SDDCFQDN SHORTNAME localhost

• Re-run the health check and make sure DNS health check is GREEN using following command 

/opt/vmware/sddc-support/sos --health-check --domain-name ALL --skip-cert-check

Workaround 2:

1. Edit the hosts file so that it is exactly like the section below.
   
   # Begin /etc/hosts (network card version)

127.0.0.1 localhost.localdomain
127.0.0.1 localhost
127.0.0.1 photon
# End /etc/hosts (network card version)
   
   
2. Save and exit the file (press ESC, type :wq, press Enter).
3. Verify the changes:
   
   cat /etc/hosts
   
   
4. Restart the services on the SDDC manager: 
   
   /opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh