Support for installing 3rd-Party applications on TCA-Managed management clusters

Products

• VMware Telco Cloud Automation
• VMware Telco Cloud Platform

Issue/Introduction

• Installing third-party applications on the management cluster is not supported.
• The only applications authorized and supported to run on the management cluster are the native add-ons explicitly provided and validated by TCA.

Environment

TCA: 2.3, 3.1, 3.1.1, 3.2, 3.3, 3.3.0.1, 3.4, 3.4.0.1

TCP: 3.0, 4.0, 4.0.1, 5.0, 5.0.1, 5.0.2, 5.1, 5.1.1

Cause

• Installing third-party applications introduces significant risks, including:

  • Resource Contention and Exhaustion: Management clusters are precisely sized to run critical control plane services. Third-party applications can consume unpredictable amounts of CPU, memory, and storage, potentially starving essential TCA components and leading to cluster instability or failure.

  • Destructive Upgrade Paths: The lifecycle of a management cluster is rigidly controlled by TCA. During a cluster upgrade, any manually installed third-party applications and configurations will be removed.

  • Lifecycle Management (LCM) Conflicts: TCA automates the patching, scaling, and upgrading of the cluster. External applications may introduce conflicting Kubernetes API versions, custom resource definitions (CRDs), or webhooks that can block, corrupt, or fail official TCA lifecycle workflows.

  • Supportability and "Noisy Neighbor" Troubleshooting: If a management cluster experiences performance degradation, isolating the root cause becomes incredibly difficult if unsupported software is running alongside core services.

  • Security and Compliance Vulnerabilities:  Introducing unverified third-party software  can potentially introduce vulnerabilities or unauthorized access to the underlying infrastructure control plane.

Resolution

• To ensure the stability, security, and predictability of our infrastructure, the management cluster must remain isolated from external software.

• Any operational tools, logging agents, or monitoring solutions required by the business should be deployed exclusively on dedicated workload clusters, where application hosting is fully supported and isolated from the core management plane.

Additional Information

Supported TCA Add-Ons and their uses:

• antrea-tca-addon: Manages the core container networking, security policies, and traffic encapsulation (CNI) for the cluster.

• vsphere-csi: Enables the cluster to dynamically provision and manage persistent block storage directly from vSphere datastores.

• nfs-client: Automatically provisions Kubernetes persistent storage volumes using an external NFS network file server.

• harbor: Integrates a secure, private container image registry to store and manage the cluster's application images.

• multus: Allows Kubernetes pods to attach to multiple network interfaces simultaneously, which is critical for complex telco network functions.

• systemsettings: Configures baseline cluster infrastructure parameters, such as administrative passwords and centralized syslog logging servers.

• load-balancer-and-ingress-service (AKO): Uses the Avi Kubernetes Operator to automate external load balancing and ingress routing for applications.

• Prometheus: Provides cloud-native monitoring, metric collection, and alerting capabilities for the cluster's health and performance.

• fluent-Bit: Collects, processes, and forwards cluster and application log data to external logging destinations like syslog.

• whereabouts: Dynamically assigns cluster-wide IP addresses to pods across multiple nodes, typically used in tandem with Multus.

• cert-manager: Automates the creation, management, and renewal of SSL/TLS certificates within the cluster.

• velero: Manages the backup, disaster recovery, and restoration of cluster states and persistent volume data.

• TKG standard extension: Allows users to deploy and manage additional packaged Tanzu Tanzu Kubernetes Grid extensions via the Tanzu CLI.