Collecting log bundles from Telco Cloud Automation (TCA) 1.x via CLI

Products

• VMware
• VMware Telco Cloud Automation

Issue/Introduction

Provide a method to collect log bundles on the TCA and/or TCA-CP via the CLI when the UI method is not working correctly.

Symptoms:
The recommended method to obtain TCA logs is from the TCA UI, however, in some cases this operation may encounter timeouts or other issues due to log size or other factors. In this case, logs can be obtained via the TCA and/or TCA-CP Command Line Interface.

The one exception to collecting logs from the UI is for TCA 1.8 and specifically Kubernetes log collection. In 1.8, a request to collect Kubernetes logs via the UI or the Support Bundle Script, will attempt to download Kubernetes logs for all clusters, this will negatively impact the TCA-CP.
It is not recommended to collect Kubernetes logs via the UI, instead it is recommended to download TCA logs using the Support Bundle Script without selecting the Kubernetes logs option and as a second step use the Central CLI to download the Kubernetes logs.

Environment

VMware Telco Cloud Automation 1.8
VMware Telco Cloud Automation 1.9

Cause

Resource exhaustion on the TCA/TCA-CP can happen due to log size or other factors may prevent the UI from being used to obtain the TCA log bundles.

Resolution

Collecting log bundles from Telco Cloud Automation via CLI

Collecting TCA Log Bundle in TCA 1.8 Using the Support Bundle Script
Collecting Log Bundles in TCA 1.9 Using the Support Bundle Script
Collect Kubernetes Logs for Management Clusters in TCA 1.8 using the TCA Central CLI (CCLI)
Collect Kubernetes Logs for Workload Clusters in TCA 1.8 using the TCA Central CLI (CCLI)
Collect Kubernetes Logs for Management Clusters in TCA 1.9 using the TCA Central CLI (CCLI)
Collect Kubernetes Logs for Workload Clusters in TCA 1.9 using the TCA Central CLI (CCLI)

Collecting TCA Log Bundle in TCA 1.8 Using the Support Bundle Script

Collecting logs via the TCA UI calls the same script as noted here. It is recommended to obtain the TCA logs via the UI unless the UI is not accessible or there are issues when collecting logs via the UI. 

In TCA 1.8, the support bundle script collects the Kubernetes logs for all clusters which may adversely impact the log collection process. For this reason, it is recommended to collect the Kubernetes logs using the TCA Central CLI documented later in this document. The core TCA logs, the crash log, mongo database dump can be collected with this script.

1. SSH into the TCA/TCA-CP appliance.
2. Switch to root and provide the root password:

su -

3. Use the change directory cd command to traverse to the following directory:

cd /opt/vmware/bin

4. Run the following command to collect the logs:

./techsupport.pl <fileName> <true/false> <true/false> <true/false> <true/false>

Where the 1st <true/false> is to collect the Core TCA logs.
2nd <true/false> is to collect the Crash Log (true requires that Core TCA logs are collected).
3rd <true/false> is to collect Mongo Database dump.
4th <true/false> is to collect all of the Kubernetes Cluster logs. In 1.8 there is no option to selectively collect logs for some clusters and not others. It is not recommended to collect all logs via this script.

Example to collect all the logs (except Kubernetes):

./techsupport.pl VMware-Telco-Cloud-Automation-tech-support-31072021 true true true false

5. Change directory to: /tmp/techsupport/ to find the tech support bundle.
6. Secure copy the file out of the TCA/TCA-CP.

Collecting Log Bundles in TCA 1.9 Using the Support Bundle Script

1. SSH into the TCA/TCA-CP appliance.
2. Switch to root by using the su command and provide the root password:

su -

3. Use the change directory cd command to traverse to the following directory:

cd /opt/vmware/bin

4. Run the following command to collect the logs:

./techsupport.pl <fileName> <true/false> <true/false> <true/false> <true/false> <comma separated cluster IDs - Provide this only if "Collecting Kubernetes Cluster Logs" is true>

Where the 1st <true/false> is to collect the Core TCA logs.
2nd <true/false> is to collect the Crash Log (true requires that Core TCA logs are collected).
3rd <true/false> is to collect Mongo Database dump.
4th <true/false> is to collect the Kubernetes logs for the clusters specified. It is not recommended to collect the logs for all clusters.
Provide the ID of the Kubernetes Cluster - you can reference the steps later in this document to obtain the Kubernetes cluster ID for your version of TCA

Example - exporting the TCA logs and the Kubernetes logs for 2 clusters: 

./techsupport.pl VMware-Telco-Cloud-Automation-tech-support-31072021 true true true true "78401444-b4f6-4e9f-893f-9277a6ac7ce5,88b0382f-ff40-4118-8d9b-b6e6fd1640aa"

5. Change directory to: /tmp/techsupport/ to find the tech support bundle.
6. Secure Copy the file out of the TCA/TCA-CP.

Collecting Kubernetes Log Bundles in TCA 1.8 using the TCA Central CLI (CCLI)

The following procedures allow you to use the TCA Central CLI to collect the Kubernetes log bundles for specific management and workload clusters in TCA 1.8.

Collect Kubernetes Logs for Management Clusters in TCA 1.8 using the TCA Central CLI (CCLI)

This procedure will collect the logs for the Management cluster specified.

1. SSH into the TCA-CP appliance where the backing Management cluster is deployed.
2. Switch to root by using the su command and provide the root password. 
3. Change directory to /tmp:

cd /tmp

4. Start the Central CLI with the following command:

ccli

5. List the management clusters and note the Index ID:

list mc

6. Go into the context of the management cluster using the Index ID:

go 0

7. Collect the tech support bundle for the selected management cluster:

techsupport

8. The logs will be placed in /tmp/<TimeStamp>.tar.gz. Secure copy them from this location.

Collect Kubernetes Logs for Workload Clusters in TCA 1.8 using the TCA Central CLI (CCLI)

This procedure will collect the logs for the Workload clusters specified in TCA 1.8.

1. SSH into the TCA-CP appliance where the backing Management cluster is deployed.
2. Switch to root by using the su command and provide the root password. 
3. Change directory to: /tmp

cd /tmp

4. Start the Central CLI with the following command:

ccli

5. List the management clusters and note the Index ID:

list mc

6. Go into the context of the management cluster using the Index ID:

go 0

7. List the workload clusters:

list wc

8. Go into the context of the workload cluster using the Index ID, in this example we are going into the workload cluster with an Index ID of 1:

go 1

9. Collect the tech support bundle for the selected workload cluster:

techsupport

10. The logs will be placed in /tmp/tech-support-<TimeStamp>.tar.gz. Secure copy them from this location.

Collecting Kubernetes Log Bundles in TCA 1.9 using the TCA Central CLI (CCLI)

The following procedures allow you to use the TCA Central CLI to collect the Kubernetes log bundles for specific management and workload clusters in TCA 1.9

Collect Kubernetes Logs for Management Clusters in TCA 1.9 using the TCA Central CLI (CCLI)

This procedure will collect the logs for the Management Cluster specified.

1. SSH into the TCA-CP appliance where the backing Management cluster is deployed.
2. Switch to root by using the su command and provide the root password. 
3. Change directory to: /tmp

cd /tmp

4. Start the Central CLI with the following command:

ccli set mode interactive

5. List the management clusters and note the Index ID:

list mc

6. Go into the context of the management cluster using the Index ID:

go 0

7. Collect the tech support bundle for the selected management cluster:

techsupport

8. The logs will be placed in /tmp/tech-support-<time_stamp>.tar.gz. Secure copy them from this location.

Collect Kubernetes Logs for Workload Clusters in TCA 1.9 using the TCA Central CLI (CCLI)

This procedure will collect the logs for the Workload clusters specified in TCA 1.9.

1. SSH into the TCA-CP appliance where the backing Management cluster is deployed.
2. Switch to root by using the su command and provide the root password. 
3. Change directory to: /tmp

cd /tmp

4. Start the Central CLI with the following command:

ccli set mode interactive

5. List the management clusters and note the Index ID:

list mc

6. Go into the context of the management cluster using the Index ID:

go 0

7. List the workload clusters:

list wc

8. Go into the context of the workload cluster using the Index ID, in this example we are going into the workload cluster with an Index ID of 1:

go 1

9. Collect the tech support bundle for the selected workload cluster:

techsupport

10. The logs will be placed in /tmp/tech-support-<time_stamp>.tar.gz. Secure copy them from this location.