Troubleshooting VMware cloud director availability NTP Issue.
search cancel

Troubleshooting VMware cloud director availability NTP Issue.

book

Article ID: 369161

calendar_today

Updated On:

Products

VMware Cloud Director

Issue/Introduction

  • When configuring a new outgoing replication, you see the following error in the Replications Tasks view of the Cloud Director Availability portal:
  • Assuming task '9XXXXXXX-0XX8-4XXf-bXX7-1cXXXXXXX69' failed, because it's status did not update in a timely fashion. 

    In /opt/vmware/h4/cloud/log/cloud.log on the Cloud Director Replication Management Appliance on the recovery site, you see a similar message:2019-04-DEBUG - [UI_/plugins/Vk13YXJl/h4/outgoing-replications/Provider_Site/vapp_fXXXX7-dXXa-41XX-bXXX-4XXXXXXXXdd_K1_4t]    [job-3] com.vmware.h4.jobengine.JobEngine        : Suspending execution for task 9XXXXXXX-0XX8-4XXf-bXX7-1cXXXXXXX69.
    DEBUG - [UI_/plugins/Vk13YXJl/h4/outgoing-replications/Provider_Site/vapp_fXXXX7-dXXa-41XX-bXXX-4XXXXXXXXdd_K1_4t] [job-3] com.vmware.h4.jobengine.JobEngine        : Suspending execution for task 3XXXXXX2-bXX3-4XXe-aXX7-cXXXXXXXXXX0
    WARN - [1XXXXX41-0XXe-4XXf1-9XXX-3XXXXXXX18b] [c4-scheduler-2] com.vmware.task.rest.client.TaskMonitor  : Task bXXXXX9-0XX8-4XXf-bXX7-1XXXXXXXX9 has timed out (it hasn't been updated in 60000 msec)
    ERROR - [UI_/plugins/Vk13YXJl/h4/outgoing-replications/Provider_Site/vapp_fXXXX7-dXXa-41XX-bXXX-4XXXXXXXXdd_K1_4t] [c4-scheduler-2] com.vmware.h4.jobengine.JobExecution     : Task 9XXXXXXX-0XX8-4XXf-bXX7-1cXXXXXXX69 (WorkflowInfo{type='start', resourceType='vmReplication', resourceId='C4-0XXXXX6f-9XX5-4XXe-aXX8-c4XXXXXXXX03', isPrivate=false, resourceName='null'}) has failed
    com.vmware.vdr.error.exceptions.TaskMonitoringTimeOutException: Assuming task 'bXXXXX9-0XX8-4XXf-bXX7-1XXXXXXXX9' failed, because it's status did not update in a timely fashion.at sun.reflect.GeneratedConstructorAccessor146.newInstance(Unknown Source)

  • In the vCDA appliance settings you see NTP service as offline.

    In /opt/vmware/h4/cloud/log/cloud.log you see "clock Skew" errors in the cloud.log.

 

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

 

Environment

VMware Cloud Director Availability 4.x

Cause

This issue can occur when there is a time drift between the Cloud Director Availability components on the protected and recovery sites.

Resolution

To resolve this issue, follow the below steps to configure the time settings.

  • SSH into all VCDA appliances and run the following command to check the time:
    watch -n 0.1 time

Verify the time in the following components across all sites:

  • Tunnel Appliance
  • Cloud Director Replication Management Appliance
  • Replicator Appliance(s)
  • Cloud Director cells
  • vCenter Server(s)
  • Platform Services Controller
  • ESXi Hosts
  • Additionally, if there is an on-premises appliance: Run the same command in the SSH console for the on-premises appliance.
  • Ensure that the Cloud Director Availability on-premises appliance, vCenter Server, Platform services controller, and ESXi Hosts have their times synced to the same NTP source.

To check the NTP services status in vCDA run the following command:

    systemctl status systemd-timesyncd
  • To restart the NTP services run the following command: 
    systemctl restart systemd-timesyncd
Powered by Wolken Software