TCP Routes from Tiles are Pruned Intermittently Resulting in connectivity issues

Products

• VMware Tanzu Platform
• VMware Tanzu Application Service
• VMware Tanzu Application Service
• VMware Tanzu Application Service for VMs
• VMware Tanzu MySQL
• VMware Tanzu RabbitMQ

Issue/Introduction

TCP Routes from to different tiles are intermittently pruned. They disappear for a few seconds, before working again.  This impacts the VMware Tanzu MySQL Tile for VMs and any other tile that uses TCP Routes. 

This issue can be diagnosed by seeing TCP routes churn in the bosh logs for the Routing API. A log with message “routing-api.prune-routes.successfully-finished-pruning-tcp-routes” and with “rowsAffected” greater than 0 will log every 10 seconds.

{"timestamp":"2024-12-16T22:13:38.525043913Z","level":"info","source":"routing-api","message":"routing-api.prune-routes.successfully-finished-pruning-tcp-routes","data":{"rowsAffected":34,"session":"8"}}
{"timestamp":"2024-12-16T22:13:48.521488189Z","level":"info","source":"routing-api","message":"routing-api.prune-routes.successfully-finished-pruning-tcp-routes","data":{"rowsAffected":34,"session":"8"}}
{"timestamp":"2024-12-16T22:13:58.521369462Z","level":"info","source":"routing-api","message":"routing-api.prune-routes.successfully-finished-pruning-tcp-routes","data":{"rowsAffected":34,"session":"8"}}

Environment

TAS Versions with routing release 0.302.0 - 0.326.0

• 4.0.27 - 4.0.31
• 6.0.7 - 6.0.11
• 10.0.0 -10.0.1

With tiles that publish TCP routes that use routing release 0.301.0 or lower.

• Tanzu for MySQL on Cloud Foundry: v3.3.0 and earlier
• Redis for VMware Tanzu Application Service: v3.5.0 and earlier
• RabbitMQ for VMware Tanzu Application Service: v2.3.0~2.3.5, v2.4.0~2.4.1 and v10.0.0

Cause

When the TCP Router was being updated to support TLS TCP routes a new ‘host_tls_port’ DB column was added for TCP routes in the Routing API. Route registrars from routing release 0.301.0 and before do not send this information to routing API and the column defaults to `NULL`. This results in multiple entries for the same route being created with every registration interval. This leads to entries expiring, and causing routing-api to tell tcp-router to prune routes that it also tells tcp-router to make exist. This leads to flapping of the tcp route, and connectivity issues for apps that do not persist their tcp connections.

Resolution

This is fixed in routing-release 0.327.0 and above.  Do not update to affected Tanzu Application Service ( TPCF ) versions. If you are on an affected version, then upgrade to TAS 4.0.32, 6.0.12, 10.0.2, or newer. 

If you are experiencing this issue and no fix is currently available for your tile then please contact support and reference this KB.