Understanding of Kubernetes Service behavior with respect to floating IP pool in VMware PKS environment

Products

• VMware Cloud PKS

Issue/Introduction

This article provides understanding on K8s Service behavior with respect to floating IP pool.

Symptoms:

• K8S service IP address will change upon recreation of service.
• How to access deployment even if the Load-Balancer IP address changes after recreating the service with same name?

Environment

VMware PKS 1.x

Resolution

• Kubernetes Services object's purpose is to guarantee a static discovery for an application during its life. i.e. if application is updated; typically this is done by keeping the service static and updating its selectors to point to the new application. If the service is deleted its ClusterIP and LoadBalancer IP is released back into the pool it came from.

• It is not possible to assign a fixed IP to a LoadBalancer service, as mentioned in NSX-T documentation:
  https://docs.vmware.com/en/VMware-NSX-T-Data-Center/2.3/com.vmware.nsxt.ncp_kubernetes.doc/GUID-FF86F9C2-E70D-418B-AFDB-B3CF8DE106C2.html

• You can request it to be the same IP by entering the IP address in the manifest file field LoadBalancerIP, however there is no guarantee that the IP will be available as another service may have been allocated to it. If the IP is taken, service will be created but it's external IP entry will stay in Pending state, relying on static IP is not recommended.

• If you need to reach your deployments even if the Load-Balancer IP address changes after recreating the service, You can create dynamic DNS entries based on the K8s service:
  https://github.com/kubernetes-incubator/external-dns