Error: Unable to reach tca service from ingress. Please check the status of the service by accessing the cluster
search cancel

Error: Unable to reach tca service from ingress. Please check the status of the service by accessing the cluster

book

Article ID: 427740

calendar_today

Updated On:

Products

VMware Telco Cloud Platform VMware Telco Cloud Automation

Issue/Introduction

  • Users are unable to access the VMware Telco Cloud Automation Control Plane (TCA-CP) web interface on port 9443.
  • The browser displays the error: "Unable to reach tca service from ingress. Please check the status of the service by accessing the cluster".
  • Pods in the tca-cp-cn namespace (e.g., tca-api, tca-app) are in CrashLoopBackOff or Error status.
  • Attempts to run kubectl logs fail with Error from server ("log" not found). 
  • The partition /dev/mapper/vg_logs-lv_logs is at 100% utilization.

Environment

  • TCA 3.2
  • TCP 5.0.2

Cause

  • The /logs partition on the Telco Cloud Automation Control Plane (TCA-CP) appliance has reached 100% capacity.
  • When the log disk is full, the Kubelet cannot create the necessary log pipes or metadata files for container execution, causing pods to fail during initialization.
  • This is frequently caused by a large accumulation of historical logs in the /logs/retained-logs directory.

Resolution

Log in to the Telco Cloud Automation Control Plane (TCA-CP)  appliance via SSH as root.

  1. Verify the disk space usage:
     df -h /logs

  2. Identify large historical log files within the retained logs directory:
     find /logs/retained-logs -type f -size +50M

  3. Navigate to the directories containing the largest files (typically associated with the tca-cp-cn pods) and remove older log rotations.
    cd /logs/retained-logs/k8s.tca-cp-cn.tca-app-<ID>/
    rm tca.2025*
    cd kubelet.service/
    rm kubelet.2025*

  4. Verify that space has been reclaimed (utilization should ideally be below 80%):
    df -h /logs

  5. Wait several minutes for the Kubernetes scheduler to automatically restart the pods. Verify pod status:
    kubectl get pods -n tca-cp-cn

  6. Verify the Telco Cloud Automation Control Plane (TCA-CP) is reachable via the web browser and showing as "Connected" in the TCA Manager UI.

 

Additional Information

Patch TCA Nodes as this is a known issue for more details. See the following KB for more details. 
Fix: Cronjob failure in airgap environment