ARP proxy entries fails to realise on transport nodes

Products

• VMware NSX

Issue/Introduction

• ARP proxy entries added via Load Balancer or AVI configurations may fail to realise and function on a host.
• A large number of virtual services or Load Balancer VIPs are present, and the segment size exceeds a /24 subnet (i.e., more than 255 IPs exist within one segment).
• The distributed router instance on a host may not respond to ARP queries, potentially disrupting traffic flows that rely on ARP proxy (e.g., NSX Load Balancers and AVI Virtual Servers).
• Proxy ARP entries appear in the nest-db level of the host configuration when checked using the following command:
  

  /opt/vmware/nsx-nestdb/bin/nestdb-cli --cmd "get vmware.nsx.nestdb.ArpTableConfigMsg" --json --beautify

• However, they are not present when checked using net-vdr commands for realized ARP proxies on the host:
  

  net-vdr -I <VDR Instance UUID>  --lif <VDR Instance Interface UUID> --proxy-arp -l

• Example outputs when the issue is present are below, these are to show the format of the outputs. For this example we see that 192.168.0.1 is contained within the output of the nest-db command however it is not listed within the net-vdr output, meaning the configuration is present but not realised.
  
  

  /opt/vmware/nsx-nestdb/bin/nestdb-cli --cmd "get vmware.nsx.nestdb.ArpTableConfigMsg" --json --beautify  p
  
                       {
                          "ip" : "192.168.0.1",
                          "mac" : "0",
                          "type" : "PROXY_ARP_ENTRY"        
                       },
                       {
                          "ip" : "192.168.0.2",
                          "mac" : "0",
                          "type" : "PROXY_ARP_ENTRY"
                        }
  
  
  
   net-vdr -I <VDR Instance UUID>  --lif <VDR Instance Interface UUID> --proxy-arp -l
  
  
  DR <UUID> Proxy ARP Information :
  Network           Mac
  192.168.0.2                 <MAC>

• When comparing proxy ARP entries in net-vdr and nest-db, it may appear random as to how many entries are realized once more than 255 are configured. Not all entries beyond 255 are lost, but realization becomes inconsistent.

Environment

VMware NSX 
VMware NSX-T Data Center

Cause

This issue is caused by a limit of ARP proxy, which is capped at 255 ARP proxies per interface. The number of ARP proxies depends on the services using them. For example, a virtual server will create a proxy ARP entry for itself if its IP overlaps with a segment connected to a logical router. If the segments are larger than a /24 or if there are more than 255 overlapping virtual servers, this limitation can be reached.

Resolution

This feature is enhanced in VMware NSX 4.2.2, available at Broadcom downloads.

If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.