Upgrading VCF from 4.1 to 4.2 fails at pre-checks with NTP TIME SYNC CHECK
search cancel

Upgrading VCF from 4.1 to 4.2 fails at pre-checks with NTP TIME SYNC CHECK

book

Article ID: 344826

calendar_today

Updated On:

Products

VMware Cloud Foundation

Issue/Introduction

Symptoms:
Error Description:
NTP time sync check failed as VC time is not in sync with SDDC VM 

Remediation:
Check if VC is down.
Check if VC/SDDCMANAGER time is in sync with NTP server.
If logged into PSC/VC on the Appliance shell, switch to bash shell using command 'shell' and try running the command 'date +%s' on target VM BASH shell. Make sure that the SDDCMANAGER and VC time are in sync.


Troubleshooting:
-NTP Health check was GREEN 
-Checked the data "date %s" on both VC and SDDC Manager.
-Tried restarting ntp service on both SDDC-Mgr and VC side. Failure still persisted.



Environment

VMware Cloud Foundation 4.x
VMware Cloud Foundation 4.0.x

Cause

Log snippet:
lcm-debug.log:
2021-04-13T18:41:08.761+0800 DEBUG [vcf_lcm,0000000000000000,0000,precheckId=dc0d2e32-####-####-####-c2b57327bf67,resourceType=COMMON_SERVICES,resourceId=a61
54964-####-####-####-7f1a3077823f] [c.v.e.s.l.p.e.util.spi.PrimitiveUtil,pool-3-thread-1] epochTimeOnVCenter(cfbcff7a-####-####-####-3aaae06a2bd4): 1618310463000, timeInSddcVM: 1618310410

NTP_TIME_SYNC_CHECK is failing as the difference of time on vcenter and sddcVM is more than 45 sec
The VC/PSC upgrade fails sometimes if the time difference is more than the required time and hence LCM added the check. 

LCM gets all the Vcenters from the inventory and checks NTP time for each of them with SDDC VM time. If any of the Vcenter time is more than 45 secs than SDDC time, the precheck task NTP_TIME_SYNC_CHECK will fail which could be because of the network latency.

Resolution



Workaround:
  • Note: Please take a snapshot of the SDDC VM prior performing the below change.
  • Change the property value (lcm.ntp.acceptable.timedifference=150) in lcm/lcm-app/conf/application-prod.properties
Add the following entries in /opt/vmware/vcf/lcm/lcm-app/conf/application-prod.properties
##### LCM NTP Properties #####
lcm.ntp.acceptable.timedifference=150
  • Restart lcm with the command "systemctl restart lcm"