Using Markers to Select the IPAM Network for TKG Cluster API VIP Allocation in Avi Load Balancer

Products

• VMware Avi Load Balancer

Issue/Introduction

• In a vSphere with Tanzu environment, Avi Load Balancer allocates Kube-API VIPs for Supervisor and Workload Clusters using the Avi IPAM profile.
• When multiple VIP networks are configured in the IPAM profile, Avi can select any network during Kube-API VIP creation. As a result, Kube-API VIPs may be allocated from a subnet different from the one intended for the cluster.

Environment

• vSphere with Tanzu 
• Avi Load Balancer

Cause

• When the same Avi cloud is used for both Supervisor AKO and Workload Cluster AKO, only one VIP network should be configured in the Avi IPAM profile.
• This single VIP network is used for allocating Kube-API VIPs for both Supervisor and Workload Clusters.
• If additional VIP networks are added, Supervisor AKO allocates VIPs in a round-robin manner across all configured networks.
• This behavior can result in unintended Kube-API VIP allocation.
• For more details, refer to the documentation: vSphere Supervisor with VDS Networking

Resolution

• If multiple VIP networks are configured in the Avi IPAM profile and a specific subnet must be used for Kube-API VIP allocation, configure a marker on the desired VIP network in Avi.
• The marker ensures Avi uses the specified VIP network for Kube-API VIP allocation, prevents selection of other networks, and applies to both Supervisor and Workload Clusters.

Steps to Configure the Marker: 

    1. Log in to the Avi Controller leader node via the CLI.

    2. Verify the VIP network added to the IPAM profile: show network <network_name>

       

     3. Configure the VIP network, add the marker, and save the configuration: (key Supervisor_clustername values <Supervisor-Cluster-UUID>)

         

      4. Restart AKO on the Supervisor Cluster and verify that Kube-API VIPs are allocated from the intended VIP network.