Unable to reach tca service from ingress. Please check the status of the service by accessing the cluster
search cancel

Unable to reach tca service from ingress. Please check the status of the service by accessing the cluster

book

Article ID: 427740

calendar_today

Updated On:

Products

VMware Telco Cloud Automation

Issue/Introduction

  • Users are unable to access the VMware Telco Cloud Automation Control Plane (TCA-CP) web interface on port 9443.
  • The browser displays the error: "Unable to reach tca service from ingress. Please check the status of the service by accessing the cluster".

  • Pods in the tca-cp-cn namespace (e.g., tca-api, tca-app) are in CrashLoopBackOff or Error status.
  • Attempts to run kubectl logs fail with Error from server ("log" not found). 
  • The partition /dev/mapper/vg_logs-lv_logs is at 100% utilization.

Environment

3.x

Cause

  • The /logs partition on the Telco Cloud Automation Control Plane (TCA-CP)  appliance has reached 100% capacity.
  • When the log disk is full, the Kubelet cannot create the necessary log pipes or metadata files for container execution, causing pods to fail during initialization.
  • This is frequently caused by a large accumulation of historical logs in the /logs/retained-logs directory.

Resolution

Log in to the Telco Cloud Automation Control Plane (TCA-CP)  appliance via SSH as root.

  1. Verify the disk space usage:
     df -h /logs

  2. Identify large historical log files within the retained logs directory:
     find /logs/retained-logs -type f -size +50M

  3. Navigate to the directories containing the largest files (typically associated with the tca-cp-cn pods) and remove older log rotations.
    cd /logs/retained-logs/k8s.tca-cp-cn.tca-app-<ID>/
    rm tca.2025*
    cd kubelet.service/
    rm kubelet.2025*

  4. Verify that space has been reclaimed (utilization should ideally be below 80%):
    df -h /logs

  5. Wait several minutes for the Kubernetes scheduler to automatically restart the pods. Verify pod status:
    kubectl get pods -n tca-cp-cn

Verify the Telco Cloud Automation Control Plane (TCA-CP) is reachable via the web browser and showing as "Connected" in the TCA Manager UI.

 

Additional Information

Patch TCA Nodes as this is a known issue for more details :- https://knowledge.broadcom.com/external/article?articleNumber=376734

Powered by Wolken Software