[VMConAWS] High CPU usage on HCX Connector / Manager

Products

• VMware Cloud on AWS

Issue/Introduction

How to resolve high CPU on HCX Manager/Connector as a result of "/opt/vmware/bin/techsupport.pl" task consuming resources.

Symptoms:
Check if HCX is stuck in a loop compressing the support log bundles

- HCX Manager shows high CPU usage around 100% for an extended period of time such as days.
- Log collection is failing
- Checking "top" from the admin command line you can see processes "/opt/vmware/bin/techsupport.pl" consuming resources
- Log collection/compression processes may be running for multiple days

Cause

This can be caused by a bad symbolic link "kafka -> /common/logs/kafka" causing an infinite loop during compression.

Resolution

On-Prem HCX Connector:
- Login and assume root privileges.
- Check for the presence of a bad symlink:

root@hcx [ /common/logs/kafka ]# ls -al /common/logs/kafka/ | grep common/logs
lrwxrwxrwx 1 kafka kafka 18 Jun 4 2019 kafka -> /common/logs/kafka

- Remove the bad symlink:

root@hcx [ /common/logs/kafka ]# rm kafka

VMC on AWS HCX Connector:
Open a support request with VMware support. 
 

Additional Information

Impact/Risks:
This could slow down HCX Manager performance and will prevent log collection from being successful, but should not have any impact on production tunnels.