After move T1 gateway from one T0 to another T0, load balancer virtual server went down

Products

• VMware NSX

Issue/Introduction

• T1 gateway is holding load balancer. The load balancer is inline mode.
• Migrate this T1 gateway from one T0 gateway to another T0 gateway.
• Virtual server on this load balancer went down due to server pool was down. All members in this pool were down due to monitor fail.
• Server pool SNAT Translation mode is Deactivated.
• Through packet capture, T1 send monitor packet to server pool members with source ip 100.64.x.x (T1 uplink interface ip address). But servers didn't reply.

Environment

VMware NSX

Cause

Server Pool member VM has two vNics. Product application is using the second vNic.

Load balancer will use T1 uplink interface ip 100.64.x.x to perform the health check for Server Pool members. After T1 changing to connect to another T0, its uplink interface ip changes.

Server Pool members does not have route to this new ip address, so could not reply health check packets. LB fails to perform the health check, which caused server pool down and virtual server down.

Resolution

On Server Pool member, add static route like:

100.64.x.x/xx next hop <product_application_vNic_gateway_ip> interface <product_application_vNic_interface>

This makes the monitor packet receive and send through the same vNic of back end server VM.

Additional Information

Reference KB: https://knowledge.broadcom.com/external/article/321320/psc-67-ha-configuration-for-resource-vce.html