ESXi servers are experiencing time drift issues with the Active Directory domain controllers, despite being configured to use NTP servers

Products

• VMware vSphere ESXi

Issue/Introduction

• Users will not be able to login to ESXi hosts using their AD accounts.
• Log file /var/run/log/vmkernel.log will show similar entries to: 

YYYY-MM-DDTHH:MM:SS cpu11:2105964)WARNING: NTPClock: 1449: system clock stepped to 1746181017.000000000, but delta 5164701 > 172800 seconds
YYYY-MM-DDTHH:MM:SS cpu11:2105964)WARNING: NTPClock: 1457: system clock stepped to 1746181017.000000000, no longer synchronized to upstream time servers

• Log file /var/run/log/syslog.log will show similar entries to:

YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: ldap_sasl_interactive_bind_s failed with error code -2
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [netlogon] Filtering list of 6 servers with list of 0 black listed servers
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [netlogon] Filtering list of 6 servers with list of 0 black listed servers
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [lsass] Failed to sync system time [error code: 40075]
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [netlogon] CLDAP timed out: <domain-controller FQDN>
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [netlogon] CLDAP timed out: <domain-controller FQDN>
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [netlogon] Filtering list of 4 servers with list of 0 black listed servers
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [netlogon] CLDAP timed out: <domain-controller FQDN>
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [netlogon] CLDAP timed out: <domain-controller FQDN>
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [netlogon] CLDAP timed out: <domain-controller FQDN>
YYYY-MM-DDTHH:MM:SS lwsmd[2101636]: [netlogon] CLDAP timed out: <domain-controller FQDN>
YYYY-MM-DDTHH:MM:SS crond[2099039]: time disparity of 86079 minutes detected

Environment

vSphere ESXi 7.x

vSphere ESXi 8.x

Cause

The time drift occurred because the Likewise Active Directory (AD) service was configured to synchronize the ESXi host clock with the AD server, while NTP was also enabled. Running both NTP and Likewise causes an issue as both agents simultaneously (and independently) try to discipline the host clock.

Resolution

In ESXi versions prior to 7.0 Update 2 and 8.0 Update 1, when hosts are joined to the Active Directory (AD) domain, time synchronization with the AD domain is enabled by default through the Likewise service. If other time synchronization methods, such as NTP or PTP, are used at the same time, this can override them.

To disable likewise time sync with AD: 

1. Access the likewise shell:

/usr/lib/vmware/likewise/bin/lwregshell

2. Navigate to the HKEY_THIS_MACHINE\Services\lsass\Parameters\Providers\ActiveDirectory directory with this command:

cd HKEY_THIS_MACHINE\Services\lsass\Parameters\Providers\ActiveDirectory
 
3. Run this command to change the synchronization time:

set_value SyncSystemTime 0

4. Exit the shell by typing quit and pressing Enter.

5. Refresh the lsass service with this command:

/usr/lib/vmware/likewise/bin/lwsm refresh lsass

6. To verify the changes to the registry key, run this command:

/usr/lib/vmware/likewise/bin/lwregshell list_values "[HKEY_THIS_MACHINE\Services\lsass\Parameters\Providers\ActiveDirectory]" | grep -i SyncSystemTime

Upgrade Scenarios:

For ESXi version 7.x:

• Upgrading from ESXi versions prior to 7.0 Update 2, it is recommended to disable Likewise time synchronization (SyncSystemTime) if NTP or PTP is configured.
• In Greenfield deployments of ESXi 7.0 Update 2 or later, Likewise time synchronization parameter (SyncSystemTime) is disabled by default.

For ESXi version 8.x:

• Upgrading to ESXi versions to 8.0 Update 1 or later enforces to disable Likewise time synchronization (SyncSystemTime) by default.