Application Staging and Smoke Tests Fail with "This dataset does not support window functions" after upgrading TPCF/EAR to version 10.x

Products

• VMware Tanzu Platform - Cloud Foundry

Issue/Introduction

After upgrading Tanzu Platform for Cloud Foundry (TPCF)/Elastic Application Runtime(EAR) from older versions (e.g., v6.0.19) to v10.x, the smoke-tests errand fails during the application staging phase. Additionally, all standard application staging attempts across the platform fail globally.

During the cf push process, the Cloud Controller returns a generic HTTP 500 and UnknownError response:

OUT: Staging app and tracing logs...
  ERR: Unexpected Response
  ERR: Response Code: 500
  ERR: Request ID:    bb73a151-e4dd-4710-801d-e64502ea29de::68b06aa3-2fe8-4ca7-b22b-c7d9e68d853c
  ERR: Code: 0, Title: , Detail: {"errors":[{"title":"UnknownError","detail":"An unknown error occurred.","code":10001}]}
  OUT: FAILED
  [FAILED] in [It] - /var/vcap/packages/smoke-tests/src/base_test.go:50 @ 05/21/26 07:00:44.796

Environment

Tanzu Platform for Cloud Foundry (TPCF)/Elastic Application Runtime(EAR)  v10.x

External System Database (CCDB)

External Database Engine: MySQL v5.7.x

Cause

This issue occurs because the environment is running an unsupported, outdated version of MySQL for the Cloud Controller Database (CCDB).

In TPCF 10.x, the modernized Cloud Controller architecture relies on the Sequel Ruby database library to execute advanced backend queries. These queries require SQL window functions, which are only available in MySQL 8.0 and later. MySQL 5.7 cannot process these queries.

When the Cloud Controller attempts to stage an application, the database rejects the query, causing an immediate crash. This can be verified by checking the Cloud Controller logs (/var/vcap/sys/log/cloud_controller_ng/cloud_controller_ng.log) for the corresponding Request ID, which will reveal the underlying database engine failure:

{"errors":[{"title":"UnknownError","detail":"This dataset does not support window functions","code":10001}]}

Resolution

To resolve this issue, the external database must be upgraded to a supported MySQL 8.0 version, and the Cloud Controller connections must be refreshed.

Step 1: Upgrade the External Database
As per the official Broadcom Breaking Changes documentation, support for external MySQL 5.7 has been removed.

Upgrade your external database instance from MySQL 5.7 to MySQL 8.0.x (e.g., 8.0.46).

Step 2: Refresh Cloud Controller Connections
Immediately following the major version upgrade of the database, the Cloud Controller might experience stale connections or Mysql2::Error::ConnectionError timeouts in the cloud_controller_ng.log due to the database going offline and changing states during the upgrade.

To clear the connection pool and force the platform to reconnect to the newly upgraded database:

Ensure the external MySQL database server is fully online and accepting connections.

Use BOSH to SSH into the cloud_controller VMs and restart the monitoring processes:

bosh -d <cf-deployment-name> ssh cloud_controller -c "sudo monit restart all"

Once the processes restart and successfully connect to the MySQL 8.0 database, application staging and the smoke-tests errand will complete successfully.