Spring Cloud Dataflow instances down

Products

• VMware Tanzu Application Service

Issue/Introduction

The dashboard UI of spring cloud dataflow instances is not accessible.

The browser shows an error similar to:

There was an unexpected error (type=Internal Server Error, status=500)

The broker log shows an exception similar to:

nested exception is reactor.core.publisher.Sinks$EmissionException: 
Spec. Rule 1.3 - onSubscribe, onNext, onError and onComplete signaled to a Subscriber MUST be signaled serially.] 
with root cause
reactor.core.publisher.Sinks$EmissionException: Spec. Rule 1.3 - onSubscribe, 
onNext, onError and onComplete signaled to a Subscriber MUST be signaled serially.
at reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:56) ~[reactor-core-3.4.39.jar:3.4.39]

Environment

Tanzu application service 4.x

Spring Cloud Dataflow 1.14.x

Cause

The broker fails to obtain a new access token due to an issue with cf-java-client which may hold on to stale refresh token when UAA replies with 500.

Resolution

The problem is resolved on cf-java-client v5.13 and Spring Cloud Dataflow v1.14.5 

As a workaround, a restart of the broker can fix the problem.