Error: Failed to validate NTP configuration due to ESXi DNS mismatch

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• NTP configuration through SDDC Manager fails with the error:  "Failed to validate NTP configuration. Validate input specification as new NtpConfiguration to VMware Cloud Foundations."
  
  
• The ESXi hosts are not in maintenance mode on the vCenter server.

Via API:

Message: Error occurred while validating details of entities.
Remediation Message: Please make sure all the entities are reachable and ACTIVE.

• Logs in SDDC manager : /var/log/vmware/vcf/operartionsmanager/operationsmanager.log

YYYY-MM-DDTHH:MM:SS ERROR [vcf_om,xxxxxxxxxxxxxxxxxxxxxxxxxxx,zzzz] [c.v.v.s.s.v.SystemConfiguratorEntitiyStateValidator,om-exec-18] Error occurred while checking the maintenance mode state for the host <ESXi Host FQDN>.
com.vmware.vcf.systemconfigurator.error.SystemConfiguratorException: Host <ESXi Host FQDN> is in MAINTENANCE mode
        at com.vmware.vcf.systemconfigurator.services.validators.SystemConfiguratorEntitiyStateValidator.checkHostMaintenanceMode(SystemConfiguratorEntitiyStateValidator.java:389)
        at com.vmware.vcf.systemconfigurator.services.validators.SystemConfiguratorEntitiyStateValidator.validate(SystemConfiguratorEntitiyStateValidator.java:108)
        at com.vmware.vcf.systemconfigurator.services.validators.SystemConfigurationsValidator.lambda$validateEntitiesState$0(SystemConfigurationsValidator.java:111)
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
        at com.vmware.vcf.common.tracing.TraceRunnable.run(TraceRunnable.java:59)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
YYYY-MM-DDTHH:MM:SS ERROR [vcf_om,xxxxxxxxxxxxxxxxxxxxxxxxxxx,zzzz] [c.v.v.s.s.v.SystemConfiguratorEntitiyStateValidator,om-exec-18] Entity state validation failed
com.vmware.vcf.systemconfigurator.error.SystemConfiguratorException: Error occurred while checking the maintenance mode state for the host <ESXi Host FQDN>.
        at com.vmware.vcf.systemconfigurator.services.validators.SystemConfiguratorEntitiyStateValidator.checkHostMaintenanceMode(SystemConfiguratorEntitiyStateValidator.java:413)
        at com.vmware.vcf.systemconfigurator.services.validators.SystemConfiguratorEntitiyStateValidator.validate(SystemConfiguratorEntitiyStateValidator.java:108)
        at com.vmware.vcf.systemconfigurator.services.validators.SystemConfigurationsValidator.lambda$validateEntitiesState$0(SystemConfigurationsValidator.java:111)
        at java.base/java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1768)
        at com.vmware.vcf.common.tracing.TraceRunnable.run(TraceRunnable.java:59)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
YYYY-MM-DDTHH:MM:SS DEBUG [vcf_om,xxxxxxxxxxxxxxxxxxxxxxxxxxx,zzzz] [c.v.v.s.u.SystemConfigErrorLocalizationUtil,om-exec-18] Locale from LocaleContextHolder - en_US

Environment

VCF 9.x

SDDC 5.x

Cause

This issue occurs due to a mismatch between the DNS name of the ESXi host recorded in SDDC Manager and the DNS name actually configured on the ESXi host.

Resolution

To successfully set the NTP configuration on the SDDC Manager, ensure the ESXi host's hostname is consistent across both the SDDC and the host itself.

• Validate the hostname on the ESXi host by logging via SSH as the root user.

esxcli system hostname get

• Validate the ESXi host details from the ESXi MOB page. Note :Enable Mob for the ESXi host - The Managed Object Browser is disabled by default in vSphere 6.x
  
  https://<ESXi FQDN>/mob/?moid=ha-searchindex&method=findByDnsName (Provide the details below and proceed with Invoke method)
  
  
  • datacenter - empty
dnsName - <ESXi FQDN>
vmSearch - false
    
    
    
    
  • From SDDC Manager verify the ESXI host FQDN to make sure they match.
    
    • Inventory > Hosts > All Hosts > FQDN
      
      
• In case of a mismatch of hostname ,set the Hostname and Domain. Note: Apply the correct FQDN as specified by SDDC manager.

  esxcli system hostname set --fqdn=<esx_host_fqdn>

  Alternatively, you can set the host and domain separately:

  esxcli system hostname set --host=<esx_short_name> --domain=<domain_name>

• Re-configure the NTP:
  
  
  • Option 1:
    • Configure NTP via SDDC Manager UI
      • Administration > Network Settings > NTP Configuration > Edit
        
        
  • Option 2:
    • Configure NTP via API
      • Developer Center>/v1/system/ntp-configuration
      • Create a JSON specification with the following content in a text editor.
        • {
    "ntpServers": [
        {
            "ipAddress": "<ntp fqdn/ip>"
        }
    ]
}
      • Execute

Additional Information

NTP configuration may also fail if SDDC Manager is unable to validate the maintenance mode state of hosts that are already in maintenance mode. In such cases, identify the host from the logs, exit it from maintenance mode, and then reconfigure NTP.

Ensure bi-directional ICMP traffic is allowed between SDDC Manager and the DNS/NTP servers.