Exclude Master/Any node from pool members in NodePort Deployment

Products

• VMware Tanzu Kubernetes Grid Management
• VMware Avi Load Balancer

Issue/Introduction

It might not be desirable to have all the nodes of a Kubernetes cluster to participate in becoming server pool members, hence key/value is used as a label-based selection on the nodes in Kubernetes to participate in NodePort.

Environment

VKS cluster with AKO and Avi Load balancer

Cause

When key/value pair is not provided all the nodes including master nodes are included as pool members. The below steps can be executed to excluded desired nodes from the pool members in a NodePort deployment.

Resolution

1. Add a label to all the worker nodes.

  Example: node.kubernetes.io/exclude-from-external-load-balancers=true

   Command: 

   kubectl label nodes <node_name> node.kubernetes.io/exclude-from-external-load-balancers=true --overwrite

    2. Edit the config map. 

    kubectl edit cm avi-k8s-config -n avi-system

  Add/Edit the below parameters in the avi-k8s-config config map

     nodeKey: node.kubernetes.io/exclude-from-external-load-balancers
     nodeValue: "true"

:wq

   

     3. Delete the ako pod. 

    kubectl delete ako-0 -n avi-system

         
     4. Validate

         Check if ako pod is running.

     kubectl get pod -n avi-system

          Check the server tab of the pool (Avi Controller UI), the master nodes should not be available.

      5. Revert after activity

             Edit the config map

    kubectl edit cm avi-k8s-config -n avi-system

  Delete the below parameters in the avi-k8s-config config map

     nodeKey: node.kubernetes.io/exclude-from-external-load-balancers
     nodeValue: "true"


:wq

   

              Delete AKO Pod

    kubectl delete ako-0 -n avi-system

          Repeat stop (4) to confirm if ako pod is running and the master nodes is available in the pool as backend servers.

 

      6. Revert/Remove the labels.

           Revert the labels

kubectl label nodes <node_name> node.kubernetes.io/exclude-from-external-load-balancers=false

           Remove the labels

kubectl label nodes <node_name> node.kubernetes.io/exclude-from-external-load-balancers-

 

Additional Information

Tunables for AKO