SCS client dependencies v1.6.1-1.6.3 doesn't re-register service after heartbeat 404 failure

Products

• Support Only for Spring

Issue/Introduction

Client applications that are bound to Spring Cloud Services (SCS) Service Registry (p-service-registry) service instances send heartbeat requests on a periodic basis.

However, when using SCS client dependencies v1.6.1-1.6.3, after a network outage is repaired, the client application doesn’t re-register itself and it keeps sending heartbeat requests and getting a 404 response. In order to restore expected behavior, the client application must be restarted.

Resolution

Client applications that are bound to Spring Cloud Services (SCS) Service Registry (p-service-registry) service instances send heartbeat requests on a periodic basis. This is expected behavior for client applications using SCS client dependencies while a Service Registry service instance is down.

The problematic behavior can be confirmed on the eureka instance by using the following commands: `cf target -o p-spring-cloud-services -s instances` and `cf logs eureka-GUID`:

2018-09-27T21:45:11.70+0200 [RTR/0] OUT eureka-4a0b80cc-cf04-4998-8573-295e0ab86c0b.cfapps-17.haas-59.pez.pivotal.io - [2018-09-27T19:45:11.692+0000] "PUT /eureka/apps/PRODUCER-DEMO/producer-demo.cfapps-17.haas-59.pez.pivotal.io:4f4ac0c1-e2be-437a-5c29-8629?status=UP&lastDirtyTimestamp=1537372442158 HTTP/1.1" 404 0 0 "-" "Java-EurekaClient/v1.4.12" "10.193.82.250:48122" "10.193.82.13:61020" x_forwarded_for:"10.193.82.44, 10.193.82.250" x_forwarded_proto:"https" vcap_request_id:"a6e49f8e-aef4-43d4-7fac-9aba04bb916e" response_time:0.007494861 app_id:"19da8056-439c-4d9e-b41c-07b1f3461356" app_index:"0" x_b3_traceid:"ddde07e2d1b10432" x_b3_spanid:"ddde07e2d1b10432" x_b3_parentspanid:"-"

Usually a failed heartbeat (PUT request) should be followed by registration (POST), because the service entry is no longer available on the eureka instance. 

2018-09-27T21:30:07.80+0200 [RTR/0] OUT eureka-4a0b80cc-cf04-4998-8573-295e0ab86c0b.cfapps-17.haas-59.pez.pivotal.io - [2018-09-27T19:30:07.700+0000] "POST /eureka/apps/PRODUCER-DEMO HTTP/1.1" 204 1096 0 "-" "Java-EurekaClient/v1.4.12" "10.193.82.250:41782" "10.193.82.13:61020" x_forwarded_for:"10.193.82.44, 10.193.82.250" x_forwarded_proto:"https" vcap_request_id:"74ac7064-376f-412a-5ed1-753011f1e412" response_time:0.101750478 app_id:"19da8056-439c-4d9e-b41c-07b1f3461356" app_index:"0" x_b3_traceid:"82929c21cb34dd95" x_b3_spanid:"82929c21cb34dd95" x_b3_parentspanid:"-"

However, when using SCS client dependencies v1.6.1-1.6.3, after a network outage is repaired, the client application doesn’t re-register itself and it keeps sending heartbeat requests and getting a 404 response. In order to restore expected behavior, the client application must be restarted.

SCS client dependencies v1.6.1–1.6.3 have an issue that can prevent a client app from connecting to a Spring Cloud Services service instance after losing an existing connection. This is happening because the Jersey support has been removed in SCS client dependency version 1.6.1.

This issue has been fixed in SCS client dependency v1.6.4 and later. VMware Tanzu recommends the use of version 1.6.4 or later.