Problems with DB connections leave the credhub VM inoperative

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Apps with config-server dependency can’t initialize successfully. As a result, the following error message is outputted on logs credhub/credhub.log: 

org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30000ms.at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:82) ~[spring-jdbc-5.3.10.jar!/:5.3.10]at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:646) ~[spring-jdbc-5.3.10.jar!/:5.3.10]at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:713) ~[spring-jdbc-5.3.10.jar!/:5.3.10]

ERROR --- DefaultExceptionHandler: An application error occurred. Please contact your CredHub administrator. 294425 2022-01-31T08:13:35.294Z [https-jsse-nio-8844-exec-23]

Environment

Product Version: 2.11

Resolution

As a workaround restart credhub VM that shows error messages.

 

bosh -d CF_DEPLOYMENT ssh credhub “sudo /var/vcap/bosh/bin/monit retart credhub”