DFW rules may not match as expected for a brief period following a vMotion

Products

• VMware NSX

Issue/Introduction

• A VM’s traffic may not match the expected DFW rule for a few seconds following a vMotion.
• This may be a scaled environment or have a large DFW config e.g. large Groups.
• Some VMs may have multiple IP addresses or have had IP addresses changed in the past.

Environment

VMware NSX 
VMware NSX-T Data Center

Cause

VMware NSX discovers the IP address bindings associated with VMs. It can learn this information from VMware Tools, ARP Snooping or DHCP Snooping. After the binding discovery, this information inputs into the Realized Bindings. The Realized Binding IP information is used to implement the IP firewall at the dataplane level. In a scaled environment, it is possible there may be a delay in learning Realized Bindings as the VM initializes on the destination host following a vMotion.

This delay may result briefly in incomplete address sets at the ESX dataplane level and consequently a rule not matching as expected.

Resolution

This is a condition that may occur in a VMware NSX environment.

Workaround:

ARP Snooping by default uses TOFU. Once a binding is realized then it is permanently retained.
The default IP Discovery segment profile has an ARP Binding of 1.
This means, for any VM the first IP discovered by ARP Snooping will enter the Realized Bindings.
However, if the VM has multiple IPs or later changes IP then they will not be realized from ARP Snooping.
The permanency of ARP Snooping TOFU can be taken advantage of to avoid any vMotion related IP discovery delays.
Create a new IP Discovery segment profile and configure the ARP Binding limit to a number that will allow for all IPs configured including any past IP changes e.g. 10.
Apply this new profile to the segment.