Spring Cloud Actuators shows incorrect status for hystrix.binderHealthIndicator.status in Spring Cloud Services (SCS) 2.1.x

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

If you are using Spring Cloud Services (SCS) 2.1.x and using circuit breaker, you may run into an issue where hystrix.binderHealthIndicator.status is DOWN even though rabbit.status is UP.

For example, see the following screenshot: 

This indicator may have no impact on the circuit breaker functionality but is still be alerted by the monitoring tools. As an operator, it is in your best interest to have the right indicator status showing and not troubleshoot false alarms.

Cause

This error is most likely caused because the client apps are on a version of Spring Boot prior to v2.2.5, which uses TLS 1.1 through the spring-amqp dependency. In older versions of spring-amqp, the default version of TLS was hardcoded to v1.1, which is not supported by new JREs anymore. If you are on versions of JRE (JBP 4.38+) and have pre-provisioned RabbitMQ file that do not support older TLS versions, then restaging one of the client apps would result in this error.

Environment

Product Version: 2.9

Resolution

To resolve this issue, change the client apps to use Spring Boot 2.2.5, which pulls in the latest spring-amqp dependency. 

Workaround

To quickly work around this issue, do one of the following:

• Set spring.rabbitmq.ssl.algorithm=TLSv1.2 in the property files and restage the client apps.
• Set management.health.binders.enabled to false on the client app.

Note: This approach does disable health check for all binders and may not fit the requirements in a production environment.