Gorouter does not prune stale routes when SSL Certificate validation is disabled in Tanzu Application Service (TAS)

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

When a foundation has SSL Certificate validation disabled for a Tanzu Application Service (TAS) foundation, Gorouters are unable to prune stale routes. This persists even if you have mutual TLS app identity verification enabled: Configure Validation of App Instance Identity with TLS.

This issue is fixed in TAS versions 2.10.6+, 2.9.14+, 2.8.20+, and 2.7.26+.

Environment

Product Version: 2.8

Resolution

Gorouter uses SSL validation in order to prune stale routes. There is a setting in the TAS Tile > Networking called “Disable SSL certificate validation for this environment”. When this option is checked, Gorouter is no longer able to self prune routes when the foundation is using TLS to verify app identity.

To detect if there are stale routes on Gorouter, follow the steps below:
 

1. `bosh ssh` into any Gorouter VM

2. Execute this command:

`sudo /var/vcap/jobs/gorouter/bin/retrieve-local-routes | jq '[to_entries[].value[] | {address: .address, private_instance_id: .private_instance_id}] | unique | group_by(.address) | map(select(length>1))'`

If the above command returns [], there are no stale routes.

If it returns json similar to the following, a single address is associated with multiple private_instance_id, there are stale routes. 

[
  [
    {
      "address": "10.255.255.154:61001",
      "private_instance_id": "7d5621ee-####-####-4233-561c"
    },
    {
      "address": "10.255.255.154:61001",
      "private_instance_id": "1260e304-####-####-523e-0c22"
    }
  ],
  [
    {
      "address": "10.255.255.154:61007",
      "private_instance_id": "360ed27e-####-####-453e-4557"
    },
    {
      "address": "10.255.255.154:61007",
      "private_instance_id": "c3023223-####-####-7c13-0312"
    }
  ]
  ...
]

To work around the issue, restart Gorouter jobs on all Gorouter VMs to clean up stale routes.

To fix this issue, uncheck the option to “Disable SSL Certificate validation” and Apply Changes. If the foundation is still not pruning stale routes, please see the following guide for more information: Preventing Misrouting.