TKGI cluster update failing with error: “Invalid gateway for network 'xxx': must be inside the range”

Products

• VMware Tanzu Kubernetes Grid

Issue/Introduction

Symptoms:

• When you are trying to update the TKGi cluster, it fails with the error similar to:
  Invalid gateway for network 'xxx': must be inside the range
• Any subsequent cluster create or update tasks also fails with same error.

Environment

VMware Tanzu Kubernetes Grid Integrated Edition 1.x

Cause

When updating a cluster with nodes_dns configured from network profile, TKGI fetches and constructs the network info for the cluster. It exposes an existing bug where there is an assumption that the node subnet is /24. If you provide a different prefix for node subnet, gateway calculation will be incorrect. Then cluster update/resize operation fails due to this issue.

BOSH has a policy that the cloud-config has to be correct in the manifest, so new cluster creation also fails.

Resolution

This is a known issue in TKGi 1.11.0, 1.11.1 and 1.11.2.

This issue will be fixed in 1.11.3/1.12.


Workaround:
To work around this issue:

1. delete the error cluster to avoid the BOSH issue
2. To update existing clusters: avoid updating/resizing clusters with subnet other than x.x.x.0/xx until a) upgrade to the fix versions, or b) remove nodes_dns field from networkprofile and update cluster with the new networkprofile
3. To create new clusters: a) use ip subnet with x.x.x.0/xx to create clusters, or b) configure nodes_dns through tile and remove nodes_dns field from network-profile when creating cluster