If vCenter is joined to Active Directory Domain, VC is expected to sync time from any of the Domain Controllers in the AD Domain omitting the NTP Server configured in VAMI. In case of any of the Domain Controllers' time is out of sync, you would see the time drift issue in vCenter.
In order to solve this issue, verify the time on every Domain Controllers in the Domain and correct the time on Domain Controller/s.
Workaround:
Note: This workaround will prevent vCenter logons with Domain User Accounts if the VC is configured to use Integrated Windows Authentication. Proceed with this workaround only if you are using LDAP, LDAPS, Open LDAP, or other Identity Federations on vCenter.
Leave vCenter from AD Domain by following the below steps.