Intermittent time drift causes inconsistencies with a nodepool's state and the UI in Telco Cloud Automation (TCA) 2.1.1 and older.

Products

• VMware
• VMware Telco Cloud Automation

Issue/Introduction

If there is a time difference between TCA and the TCA-CP, events from upstream or downstream, depending on the side with the time drift, may be discarded. This can cause certain CaaS nodepool or cluster/addon statuses to remain in a processing state in the TCA UI.

Symptoms:
The following are true:
The node pool backend is in the provisioned state, 
The node policy is in the normal state,
The status in the UI is stuck in the processing state.

Environment

VMware Telco Cloud Automation 2.0.1
VMware Telco Cloud Automation 2.1.1
VMware Telco Cloud Automation 2.1
VMware Telco Cloud Automation 2.0
VMware Telco Cloud Automation 1.8
VMware Telco Cloud Automation 1.9.5
VMware Telco Cloud Automation 1.9

Cause

TCA and TCA-CP may experience a minor time inconsistency that can lead to TCA ignoring timestamp-reliant events received from TCA-CP. This issue has been noticed in systems that frequently synchronize with the same NTP servers, resulting in a temporary time drift of a few seconds between them.

However, they quickly re-synchronize and the time drift may not be visible in real-time.

Resolution

This issue is resolved in TCA 2.2 by removing timestamp-based events between the systems.

Workaround:
Edit the nodepool and go through the edit wizard without making any changes.

1. Log into the TCA UI.
2. Navigate to CaaS Infrastructure and select the affected cluster.
3. Navigate to Cluster Configuration.
4. Click Edit.
5. Complete the wizard without making any changes.

After 30-40 seconds, the nodepool should come back to normal.