Recover Foundation Load Balancer (FLB) Management Network when the DHCP Server Allocates a Duplicated IP Address

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

If the FLB management network is configured to use DHCP and the DHCP server assigns a duplicate IP address to the FLB VM, the VM will still accept and use it. As a result, the FLB may report an xDS Connection Down error and transition to an unhealthy state.

Environment

• VCF 9.0
• Supervisor is deployed with Foundation Load Balancer
• Managemnt network is DHCP

Cause

The FLB VM does not perform duplicate address detection. It unconditionally accepts the IP address provided by the DHCP server, even if the address is already in use elsewhere on the network.

Resolution

The following will reconfigure the FLB network interface and trigger a new DHCP request, allowing it to obtain an IP address from the updated DHCP configuration:

1. Navigate to the vCenter Main Menu. Open the Foundation Load Balancers section from the drop-down menu.
2. Locate the unhealthy FLB instances and check the management IP.
3. Login to the FLB instance with the username vmware-system-user. The password can be reset via Change password in the UI. 
   
   1. If the management IP is still reachable from the LAN, you can SSH to the affected foundation loadbalancer instance using SSH. 
   2. If the management IP is not reachable, you will need to find the ESXi instance of the foundation loadbalancer instance and login via the ESXi web console.
4. Run ip a to check the name of the interface that has the wrong DHCP lease. It should be one of eth0, eth1 or eth2. 
5. Run networkctl reconfigure ethX to obtain a new DHCP lease. ethX should be replaced by the name from the previous step.