Container gateway deployment gives ERROR - Timed out while waiting for database to come up

Products

• CA API Gateway

Issue/Introduction

When deploying the container gateway, it started receiving an error. The gateway is deployed on an AKS cluster. It was working fine all week and all of a sudden, it started throwing SQL exceptions and complaining about being unable to connect to database. The database is deployed in Azure Flexible that can be reached using the MySQL utility and the health of the resource is fine.

Connection string being used (as found in the values file):
jdbcURL: jdbc:mysql://<HOSTNAME>.mysql.database.azure.com:3306/ssg?useSSL=true&requireSSL=true&verifyServerCertificate=false&enabledTLSProtocols=TLSv1.2

Logs during gateway install:
default/var/logs/ssg_gc.log
2025-06-13T12:11:47.390935291Z The system property 'com.l7tech.security.fips.enabled' has not been set to true in either the EXTRA_JAVA_ARGS environment variable or the system.properties file.
2025-06-13T12:11:47.390944991Z Disabling FIPS mode.
2025-06-13T12:11:47.391031190Z SSG_JVM_HEAP has been deprecated! Please use SSG_JVM_MIN_HEAP and SSG_JVM_MAX_HEAP going forward.
2025-06-13T12:11:47.392586783Z SSG_JVM_MIN_HEAP will be 6g
2025-06-13T12:11:47.392597383Z SSG_JVM_MAX_HEAP will be 6g
2025-06-13T12:11:47.392630083Z Using MySQL database jdbc:mysql://<HOSTNAME>.mysql.database.azure.com:3306/ssg?useSSL=true&requireSSL=true&verifyServerCertificate=false&enabledTLSProtocols=TLSv1.2
2025-06-13T12:11:47.392649083Z SSG_DATABASE_WAIT_TIMEOUT set to 300 seconds.
2025-06-13T12:11:47.413099188Z SSG_CLUSTER_HOST will be <HOSTNAME>
2025-06-13T12:11:47.414706780Z Creating MySQL Database
2025-06-13T12:11:47.439230366Z Waiting for one of the databases to come up...
2025-06-13T12:16:49.177970048Z ERROR - Timed out while waiting for database to come up

This article was found via a Google search:
https://knowledge.broadcom.com/external/article/232608/gateway-container-not-able-to-connect-to.html 

There is no lock showing in the table.

There's been no success when trying to connect to the container to run curl:
docker exec -it layer7-api-dev-gateway-yyyyyy /bin/bash

Error during connect: Get "http:zzzzzdockerDesktopLinuxEngine/v1.49/containers/layer7-api-dev-gateway-aaaaa/json": open //./pipe/dockerDesktopLinuxEngine: The system cannot find the file specified.

Environment

Container Gateway 11.1

Cause

DNS could not resolve the database name/host

Resolution

Run a curl command to check if the external RDS is getting resolved from the DNS. If not resolving, restart the CoreDNS pods on the AKS cluster.