Error: Failed to upload the VSAN HCL DB due to timestamp mismatch
search cancel

Error: Failed to upload the VSAN HCL DB due to timestamp mismatch

book

Article ID: 423684

calendar_today

Updated On:

Products

VMware SDDC Manager / VCF Installer

Issue/Introduction

When syncing the vSAN Hardware Compatibility List (HCL) between SDDC Manager and vCenter Server, the operation fails despite successful API communication. This issue typically occurs in VMware Cloud Foundation (VCF) environments where there is a timezone discrepancy between management components. Users may observe that the SDDC Manager UI becomes unresponsive or displays a "perpetually loading" state during the sync attempt.

  • The SDDC Manager UI hangs or remains in a "Fetching task info" state for extended periods.
  • The vSAN HCL sync fails with the following error in the vCenter UI: "Failed to upload the VSAN HCL DB. VSAN HCL DB timestamp is not updated in vCenter even after successful API"

The /var/log/vmware/vcf/lcm/lcm-debug.log

yyyy-mm-ddT11:43:54.870+0800 INFO  [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.p.i.vsan.VsanHealthService,HCL-Sync-1] For the new HCL, the timestamp is 1763007154 & jsonUpdatedTime is 1763007120
yyyy-mm-ddT11:43:54.871+0800 DEBUG [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.p.i.vsan.VsanHealthService,HCL-Sync-1] We found entry for jsonUpdatedTime, hence using jsonUpdatedTime as timestamp value for new HCL
yyyy-mm-ddT11:43:58.883+0800 WARN  [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.p.i.vsan.VsanHealthService,HCL-Sync-1] Expected HCL timestamp 1763007120 is NOT same as actual HCL timestamp 1762978320
yyyy-mm-ddT11:43:59.178+0800 INFO  [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.p.i.vsan.VsanHealthService,HCL-Sync-1] For the new HCL, the timestamp is 1763007154 & jsonUpdatedTime is 1763007120
yyyy-mm-ddT11:43:59.178+0800 DEBUG [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.p.i.vsan.VsanHealthService,HCL-Sync-1] We found entry for jsonUpdatedTime, hence using jsonUpdatedTime as timestamp value for new HCL
yyyy-mm-ddT11:44:02.646+0800 WARN  [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.p.i.vsan.VsanHealthService,HCL-Sync-1] Expected HCL timestamp 1763007120 is NOT same as actual HCL timestamp 1762978320
yyyy-mm-ddT11:44:02.953+0800 INFO  [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.p.i.vsan.VsanHealthService,HCL-Sync-1] For the new HCL, the timestamp is 1763007154 & jsonUpdatedTime is 1763007120
yyyy-mm-ddT11:44:02.953+0800 DEBUG [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.p.i.vsan.VsanHealthService,HCL-Sync-1] We found entry for jsonUpdatedTime, hence using jsonUpdatedTime as timestamp value for new HCL
yyyy-mm-ddT11:44:06.563+0800 WARN  [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.p.i.vsan.VsanHealthService,HCL-Sync-1] Expected HCL timestamp 1763007120 is NOT same as actual HCL timestamp 1762978320
yyyy-mm-ddT11:44:06.585+0800 ERROR [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.s.impl.VsanHclSyncRunner,HCL-Sync-1] Failed to update vSAN HCL in vCenter: <hostname> and cluster <cluster_name> with exception Failed to upload the VSAN HCL DB. VSAN HCL DB timestamp is not updated in vCenter even after successful API.
yyyy-mm-ddT11:44:06.585+0800 ERROR [vcf_lcm,0000000000000000,0000] [c.v.e.s.l.s.impl.VsanHclSyncRunner,HCL-Sync-1] Uploading vSAN HCL to vCenter: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx failed with exception: Failed to upload the VSAN HCL DB. VSAN HCL DB timestamp is not updated in vCenter even after successful API.
com.vmware.evo.sddc.lcm.model.error.LcmException: Failed to upload the VSAN HCL DB. VSAN HCL DB timestamp is not updated in vCenter even after successful API.

 

Environment

VMware Cloud Foundation 4.5.x

VMware Cloud Foundation 5.x

Cause

The vCenter Server and SDDC Manager reside in different timezones (other than UTC). This discrepancy causes an incorrect timestamp comparison during the vSAN HCL JSON payload update, leading to validation failures and prolonged processing times.

Resolution

To resolve this issue, the timezone for both the vCenter Server and SDDC Manager must be set to UTC.

Follow these steps:

  1. Log in to the vCenter Server Management Interface (VAMI) as root (https://:5480).
  2. Navigate to Time > Time zone and click Edit.
  3. Select UTC from the drop-down menu and click Save.
  4. Log in to the SDDC Manager appliance via SSH as the vcf user and switch to root.
  5. Update the SDDC Manager timezone to UTC using the command line or administrative interface.
  6. Reboot both the SDDC Manager and the vCenter Server for the changes to take full effect.
  7. Verify the vSAN HCL sync status in the SDDC Manager UI.

Additional Information

Powered by Wolken Software