BBR Restore fails after upgrading to TAS 3.0
Article ID: 298138
Updated On:
Products
Issue/Introduction
BBR Restore on a new installation of TAS 3.0 fails when restoring UAA if you've upgraded to TAS 3.0 release from 2.13, 2.12, or 2.11 and lower. You will see the following error message from BBR output when bbr restore is executed
Error attempting to run restore for job bbr-uaadb on backup_restore/253426a1-a4a7-4882-b65f-700840974d8b: 2023/03/22 15:54:30 MYSQL server version 8.0.30-22 ERROR 3780 (HY000) at line 25: Referencing column 'SESSION_PRIMARY_ID' and referenced column 'PRIMARY_ID' in foreign key constraint 'SPRING_SESSION_ATTRIBUTES_FK' are incompatible. 2023/03/22 15:54:30 You may need to delete the artifact-file that was created before re-running. exit status 1 - exit code 1
- The BBR Backup and Restore workflow requires an operator to recreate their TAS environment.
- In an internal DB configuration the platform creates the databases within the mysql cluster, when the platform initializes and creates DBs it leverages the default character and collation set of the mysql release.
- As part of the TAS 3.0 upgrade the Mysql version packaged in Internal DB was upgraded from 5.7 to 8; this change introduced a new default for character sets and collation.
BBR uses pg_dump to backup the internal databases, pg_dump preserves the character set and collation of the database its backing up.
This causes a datatype incompatibility issue when BBR attempts to load the UAA database onto a newly installed TAS 3.0 deployment.
Versions before TAS 3.0 had the default character set and collation of the database as utf8mb3 and utf8mb3_unicode_ci, in TAS 3.0 it changed to utf8mb4 and utf8mb4_0900_ai_ci.
Thus without intervention your BBR Backup is likely incompatible with TAS 3.0. The engineering teams who are relevant to this issue are working on a permanent fix, for now you can attempt the workaround of manually initializing the UAA database. At this time only UAA database has been affected by this issue, Cloud Controller does not appear to be affected; please open a ticket with VMware Tanzu Support if multiple databases are affected
Environment
Resolution
To workaround the issue we will connect to the MySQL database and initialize the DB.
1. SSH to the Mysql Database in TAS deployment
2. Connect to the Mysql process running
sudo mysql --defaults-file=/var/vcap/jobs/pxc-mysql/config/mylogin.cnf
3. Confirm the character and collation set of UAA database is set to utf8mb4 and utf8mb4_0900_ai_ci.
mysql> use uaa;
mysql> SELECT * FROM INFORMATION_SCHEMA.SCHEMATA;
mysql> SELECT @@character_set_database, @@collation_database;
+--------------------------+----------------------+ | @@character_set_database | @@collation_database | +--------------------------+----------------------+ | utf8mb4 | utf8mb4_0900_ai_ci | +--------------------------+----------------------+
mysql> SELECT SCHEMA_NAME,DEFAULT_CHARACTER_SET_NAME, DEFAULT_COLLATION_NAME FROM INFORMATION_SCHEMA.SCHEMATA WHERE SCHEMA_NAME='uaa';
+-------------+----------------------------+------------------------+ | SCHEMA_NAME | DEFAULT_CHARACTER_SET_NAME | DEFAULT_COLLATION_NAME | +-------------+----------------------------+------------------------+ | uaa | utf8mb4 | utf8mb4_0900_ai_ci | +-------------+----------------------------+------------------------+
4. If your TAS 3.0 deployment is set to use utf8mb4 and utf8mb4_0900_ai_ci then continue, if the values do not match you are likely not suffering from this specific issue. Please open a support case to investigate.
5. Drop the UAA Database and Create w/ character and collation defaults
mysql> DROP DATABASE uaa;
mysql> CREATE DATABASE uaa CHARACTER SET utf8mb3 COLLATE utf8mb3_unicode_ci;
mysql> exit
At this point you can resume running your BBR Restore foundation. Although we are dropping the UAA database this should be tolerated since there should be no critical workloads running on the platform yet.
Feedback
