Client VM downloads large file from Server fails due to traffic passing through AVI SE

Products

• VMware NSX

Issue/Introduction

• Network Topology:
  Client --- AVI SE --- T1 service interface -- T0 --- North bound routers --- Server
• Client VM connects to NSX vlan segment A. Gateway is set as the floating IP on AVI SE. On AVI SE, there was default route 0.0.0.0/0 next hop to T1 service interface. T1 service interface also connects to vlan segment A. Client VM, SE floating IP and T1 service interface IP are in the same IP subnet.
• Client VM downloads large file from Server. The traffic could transfer for a short time, then stopped and failed.
• AVI SE is configured as in-line mode, SE works as a router. In this mode, members in AVI server pool need to set gateway as SE floating IP, to make sure traffic could pass SE.
• Client VM is not a member of AVI server pool, so does not have to set gateway to AVI SE floating IP.
    

Environment

VMware NSX

Resolution

On Client VM, change gateway from SE floating IP to T1 service interface IP. This makes traffic between Client VM and Server not passing AVI SE. Client VM then could download large file from Server.