Unexpected pod restarts after a CNF lifecycle operation, occurring after upgrading to TCA 3.2
search cancel

Unexpected pod restarts after a CNF lifecycle operation, occurring after upgrading to TCA 3.2

book

Article ID: 417150

calendar_today

Updated On:

Products

VMware Telco Cloud Automation

Issue/Introduction

After upgrading TCA from version 2.3 to 3.2, customers may observe unexpected pod restarts within a namespace. These restarts typically occur after performing a CNF lifecycle management operation, such as instantiate, upgrade, or reconfigure), on a CNF that was originally deployed on TCA 2.3.

Environment

3.2

Cause

The root cause of this behavior is the introduction of the "CNF Granular Updates" feature in TCA 3.2. 

Resolution

As part of the CNF granular updates enhancement introduced in TCA 3.2, the tca-helm-release label is added under spec.template.metadata.labels to ensure it is automatically propagated to every Pod and ReplicaSet created by the CNF. This enables granular status tracking across all CNF lifecycle operations, including instantiate, terminate, rollback, scale, upgrade, and reconfigure.

Since this label resides under spec.template, any modification to it changes the Pod template hash, triggering a rollout restart of the associated Kubernetes object. However, the label is applied only once during the initial lifecycle operation (e.g., instantiate, upgrade, or reconfigure) in TCA 3.2. Subsequent or repeated lifecycle operations on the same CNF will not trigger additional restarts, as the label already exists and no further template changes occur.

This is an expected behavior.

Additional Information

This process happens in the following sequence:  
                                                                                  
1. A CNF is deployed in TCA 2.3 (without the tca-helm-release label).                                                           
2. The TCA instance is upgraded to 3.2.                                                                                             
3. An LCM operation is executed on the existing CNF.                                                         
4. TCA injects the tca-helm-release label into the resources spec.template.metadata.                                          
5. The change to the template hash causes Kubernetes to initiate a rollout restart of the pods in the StatefulSet.