Not all Worker Nodes serving ingress traffic in cluster deployed by Cloud Director Container Service Extension

Products

• VMware Cloud Director

Issue/Introduction

Symptoms:

• Not all worker nodes serve ingress traffic in Tanzu Kubernetes Grid cluster deployed by Cloud Director Container Service Extension.
• The Kubernetes Service has externalTrafficPolicy=Local set.
• Testing the nodePort on each of the worker nodes will show that only some of them serve traffic, only the nodes on which the service pods actually run.
• Testing the Load Balancer External IP of the service will intermittently fail when the NSX Advanced Load Balancer forwards the request to a node where the pod of the service does not run.

Environment

VMware Cloud Director 10.x
Cloud Provider Interface 1.6.x, earlier, and potentially newer versions.

Cause

Cloud Provider Interface (CPI) does not support externalTrafficPolicy=Local, it only supports externalTrafficPolicy=Cluster.
These symptoms will occur if the Kubernetes Service in question has externalTrafficPolicy=Local set.

Resolution

To resolve this issue change the externalTrafficPolicy option of the Kubernetes Service from externalTrafficPolicy=Local to externalTrafficPolicy=Cluster.

For example the current service configuration could be viewed with a command like the following:

kubectl get svc -n <namespace of service> <name of service> -oyaml

The service configuration could then be changed using a command such like following:

kubectl edit svc -n <namespace of service> <name of service>