Azure VM's use Host Hardware Clock for NTP instead of User-Configured NTP

Products

• Operations Manager

Issue/Introduction

As a default in Azure Linux VMs they are configured to use the Host Hardware Clock (PHC0) for NTP, as a result this could cause clock skew if the host hardware clock is out of sync. Operators in turn depend on configuring and using their own NTP servers to ensure their VM's Clock are in sync.

There was an issue where BOSH would not configure the NTP Servers on the VMs and depended on Azure's Host Hardware Clock (PHC0) for syncing time amongst VMs. VM's using pre version Xenial Stemcell 621.280 will not use the NTP server configured in Bosh tile (Operations Manager -> Bosh TILE -> Director Config -> NTP Servers)

Environment

Product Version: 2.10

Resolution

To confirm if your NTP server's are using Azure Host Hardware Clock you will SSH into your VM's and use chronyc utility to check.

Checking using Bosh SSH (pre stemcell 621.280)

ubuntu@ps-manager-2:~$ bosh -d pivotal-telemetry-om-0f3ab9e0e04286e5a590 ssh -c 'sudo chronyc tracking' -r
Using environment '10.0.4.10' as client 'ops_manager'

Using deployment 'pivotal-telemetry-om-0f3ab9e0e04286e5a590'

Task 605. Done

Instance   telemetry-centralizer/9bbc9b84-671c-4a6c-aa68-37d0bc7428bb
Stdout     Reference ID    : 80.72.67.48 (PHC0) <--- PHC0 Signifies HW Clock 
           Stratum         : 1
           Ref time (UTC)  : Thu Nov 17 19:37:12 2022
           System time     : 0.000011328 seconds fast of NTP time
           Last offset     : +0.000012041 seconds
           RMS offset      : 0.000011771 seconds
           Frequency       : 5.033 ppm slow
           Residual freq   : +0.163 ppm
           Skew            : 1.054 ppm
           Root delay      : 0.000000 seconds
           Root dispersion : 0.000012 seconds
           Update interval : 8.0 seconds
           Leap status     : Normal

-----------------------------------------

Checking using Bosh SSH (post stemcell 621.280)

ubuntu@ps-manager-2:~$ bosh -d pivotal-telemetry-om-0f3ab9e0e04286e5a590 ssh -c 'sudo chronyc tracking' -r
Using environment '10.0.4.10' as client 'ops_manager'

Using deployment 'pivotal-telemetry-om-0f3ab9e0e04286e5a590'

Task 613. Done

Instance   telemetry-centralizer/9bbc9b84-671c-4a6c-aa68-37d0bc7428bb
Stdout     Reference ID    : 40.119.6.228 (40.119.6.228) <-- time.windows.com resolved IP
           Stratum         : 4
           Ref time (UTC)  : Thu Nov 17 19:48:54 2022
           System time     : 0.000259912 seconds slow of NTP time
           Last offset     : -0.000374316 seconds
           RMS offset      : 0.000476770 seconds
           Frequency       : 7.006 ppm slow
           Residual freq   : -0.176 ppm
           Skew            : 16.972 ppm
           Root delay      : 0.064504 seconds
           Root dispersion : 0.033181 seconds
           Update interval : 64.1 seconds
           Leap status     : Normal