Error trying to create new tasks or deploy with new install of SCDF 2.11.5 in AWS EKS with embedded MariaDB instance.

Products

• VMware Tanzu Spring Runtime

Issue/Introduction

Receiving an error when trying to create new tasks or attempt to deploy with new install of SCDF 2.11.5 in AWS EKS environment using embedded MariaDB instance.

Error when creating new tasks or attempting a deploy:

Could not increment ID for BOOT3_TASK_SEQ
sequence table; nested exception is
java.sql.SQLSyntaxErrorException: (conn=144245)
Unknown column 'ID' in 'SET'

Environment

SCDF 2.11.5

AWS EKS

MariaDB 10.11.11 (MariaDB provided via Bitnami chart - docker.io/bitnami/mariadb:10.11.11-debian-12-r7)

helm install scdf oci://registry-1.docker.io/bitnamicharts/spring-cloud-dataflow --set externalRabbitmq.enabled=true --set rabbitmq.enabled=false --set externalRabbitmq.host=rabbitmq --set externalRabbitmq.username=$ACCT_USER_NM --set externalRabbitmq.password=$ACCT_USER_PWD --set server.service.type=LoadBalancer --set server.service.ports.http=9393 --set server.service.annotations=service.beta.kubernetes.io/aws-load-balancer-type:internal --set server.image.pullSecrets=['regcred']

Cause

There appears to be an issue with the schemas in MariaDB 10.11.11 that is provided in Bitnami chart docker.io/bitnami/mariadb:10.11.11-debian-12-r7.

Resolution

1. Run the following schemas in sequence:

https://github.com/spring-cloud/spring-cloud-dataflow/blob/main/spring-cloud-dataflow-server-core/src/main/resources/application-init-mariadb.yml

/schemas/mariadb/V1-dataflow.sql
/schemas/mariadb/V1-skipper.sql
/schemas/mariadb/V2-dataflow.sql
/schemas/mariadb/V2-dataflow-after.sql
/schemas/mariadb/V3-dataflow.sql
/schemas/mariadb/V4-dataflow.sql
/schemas/mariadb/V5-dataflow.sql
/schemas/mariadb/V6-dataflow.sql
/schemas/mariadb/V7-dataflow.sql
/schemas/mariadb/V8-dataflow.sql
/schemas/mariadb/V9-dataflow.sql
/schemas/mariadb/V10-dataflow.sql
/schemas/mariadb/V11-dataflow.sql
/schemas/mariadb/V12-dataflow.sql

2. Restart the server.

3. Several errors occur due to the DB not expecting to find the tables that existed.

4. Manual fix for the errors seen after running the above schemas in sequence:

drop table BATCH_JOB_SEQ;
 
CREATE SEQUENCE BATCH_JOB_SEQ START WITH 1 MINVALUE 1 MAXVALUE 9223372036854775806 INCREMENT BY 1 NOCACHE NOCYCLE ENGINE=InnoDB;
 
Drop table BATCH_JOB_EXECUTION_SEQ;
 
CREATE SEQUENCE BATCH_JOB_EXECUTION_SEQ START WITH 1 MINVALUE 1 MAXVALUE 9223372036854775806 INCREMENT BY 1 NOCACHE NOCYCLE ENGINE=InnoDB;
 
Drop table BATCH_STEP_EXECUTION_SEQ;
 
CREATE SEQUENCE BATCH_STEP_EXECUTION_SEQ START WITH 1 MINVALUE 1 MAXVALUE 9223372036854775806 INCREMENT BY 1 NOCACHE NOCYCLE ENGINE=InnoDB;
 
ALTER TABLE `BATCH_JOB_EXECUTION_PARAMS` ADD COLUMN `PARAMETER_NAME` varchar(2500) DEFAULT 1;
ALTER TABLE `BATCH_JOB_EXECUTION_PARAMS` ADD COLUMN `PARAMETER_TYPE` varchar(100) DEFAULT 1;
ALTER TABLE `BATCH_JOB_EXECUTION_PARAMS` ADD COLUMN `DATE_VAL` DATETIME DEFAULT NULL;
ALTER TABLE `BATCH_JOB_EXECUTION_PARAMS` ADD COLUMN `PARAMETER_VALUE` varchar(2500) DEFAULT 1;
ALTER TABLE `BATCH_STEP_EXECUTION` ADD COLUMN `CREATE_TIME` datetime(6) DEFAULT CURRENT_TIMESTAMP;
ALTER TABLE `BATCH_STEP_EXECUTION` MODIFY COLUMN `START_TIME` datetime(6) DEFAULT CURRENT_TIMESTAMP;