TAS v4.0 mySQL vm's showing gradual CPU consumption when utilising 'MySQL v8'

Products

• VMware Tanzu Application Service
• VMware Tanzu MySQL

Issue/Introduction

Users may experience a gradual increase in CPU resources on TAS mySQL vm(s) when utilizing MySQL v8 if an internal backing database is configured for TAS for vm's v4.0.  The gradual consumption of CPU usually occurs over a period of a few days/weeks, though can vary depending on size and demands of the environment.

An accompanying symptom to the above can be bloated Cloud Controller vm's.

Environment

TAS for VM's v4

Cause

There is a performance regression MySQL v8 due to bug:  - SELECT COUNT(*) degraded performance on 8.0 compared to 5.7.

Resolution

The fix:

The issue is being addressed MySQL Community, however this is not yet consumed in the Percona release of MySQL used by TAS.  The fix will be included in a upcoming version of mySQL, then implemented into the TAS platform.  Users can check the release notes of the mySQL version to see if fix/patch is included.

 

The Workaround:

1. Users can try to 'monit restart' services on the node to clear the issue (though is expected to eventually return, requiring another 'monit restart'.

2. Users can try one of the following workarounds to help address the issue until the above mentioned fix/patch becomes available.  For awareness, if a user decides to proceed with one of the below workarounds, it is recommended to open a ticket with support to confirm the issue is being hit and to help implement.

• Disable notifications service if it contains large sized jobs table.
• Reduce unnecessary CAPI requests to reduce the workload on mysql
• Review database tables, if any table has a considerable number of rows, for example a few millions
• Vertically scale up mysql instance with more CPU & memory resources

Additional Information

The most direct symptom of this problem is high MYSQL CPU Use Percentage KPI .