vCenter Upgrade fails with the Error: "The source vCenter Server Appliance cannot access the configured NTP servers."

Products

• VMware vCenter Server

Issue/Introduction

1. vCenter upgrade will fail in stage 2 with the below error: The source vCenter Appliance cannot access the configured NTP servers: 
   
   
   
   2. Running the date command shows a visible time difference between the vCenter and the current time on the NTP server.
   
   

        3. When running the following command:

systemctl status ntp

You may observe errors similar to the ones below:

Domain.local ntpd[1789]: Listening on routing socket on fd #20 for interface updates
Domain.local ntpd[1789]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
Domain.local ntpd[1789]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized
local systemd[1]: Started Network Time Service.

 

Cause

1. This usually happens when the ntpd or the timesyncd service is masked.
2. Provided NTP server is incorrect

Resolution

To resolve this issue, check the status of the ntpd and timesyncd services.

1. Run the below command on the vCenter to check the status:

systemctl status ntpd

Output:
● ntpd.service
   Loaded: masked (Reason: Unit ntpd.service is masked.)
   Active: inactive (dead) 
  Main PID: 18##  (code=exited, status=0/SUCCESS) ...

systemctl status systemd-timesyncd

Output:
● systemd-timesyncd.service
   Loaded: masked (Reason: Unit systemd-timesyncd.service is masked.)
   Active: inactive (dead)

The ntpd and timesyncd services are masked and currently inactive.

2. Unmask the service and then start it:

systemctl unmask ntpd
systemctl start ntpd

3. Check the status of the ntpd service and if it shows active, restart the Stage 2 Pre-upgrade check and see if the error is fixed.

systemctl status ntpd

Output:
● ntpd.service - Network Time Service
   Loaded: loaded (/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled)
   Active: active (running) since Sat yyyy-mm-dd 08:25:10 UTC; 5s ago
     Docs: man:ntpd
  Process: 49## ExecStart=/usr/bin/ntpd -g -u ntp:ntp (code=exited, status=0/SUCCESS)
 Main PID: 49## (ntpd)
    Tasks: 2 (limit: 9830)
   Memory: 1.5M
   CGroup: /system.slice/ntpd.service
           └─49## /usr/bin/ntpd -g -u ntp:ntp

Workaround:
If the issue still exists after performing the above steps, unmask the timesyncd service by running the below commands:
systemctl unmask systemd-timesyncd
systemctl start systemd-timesyncd

The Pre-upgrade check should now be completed successfully.

Note: ntpd and timesyncd should not be used in parallel, as in theory, they could pick different time servers that have a slight delay between them. This could result in the vCenter's clock experiencing periodic fluctuations. So, it is recommended to stop (systemctl stop systemd-timesyncd) the timesyncd service after the upgrade is complete.

Alternatively 

If the NTP IP address currently in use is not reachable or no longer valid, you can verify its status by using the ping command. If the NTP server is unresponsive, disable the NTP service and re-run the upgrade process.