RARP (Reverse ARP) broadcast packets are sent to the switch which the edge uplinks connected to when NSX-T Edge Nodes are rebooted/powered ON, vMotion'ed or Storage vMotion'ed
book
Article ID: 322563
calendar_today
Updated On:
Products
VMware NSX
Issue/Introduction
This article is to provide information about RARP packet behavior during VM reboot/power ON or vMotion of guest VMs belonging to a trunk VLAN
Symptoms:
The edge node uplink is configured with a trunk VLAN (0-4096) for connectivity to the physical switch.
10,000+ RARP packets are broadcast to the physical switch via the edge node uplink interface in a period of 400 milliseconds, when performing edge reboot/power ON Or vMotion/Storage vMotion of the edge node.
This leads the physical switch to detect a broadcast storm if it has a storm control configuration like "storm-control broadcast level pps 10k".
Environment
VMware NSX-T Data Center
VMware NSX-T Data Center 3.x
VMware NSX 4.x
Cause
RARP packets are sent to the edge nodes uplink physical switch, when during vMotion of an NSX-T Edge Node, so that physical switch can learn MAC address of NSX-T Edge Node's fp-ethX (VTEP) interface.
This behaviour is also during Edge Node reboot or power ON or any other operations where its vNICs are activated.
Without these RARP packets, packets destined to the NSX-T Edge Node VTEP interface could continue to be forwarded to the ESXi host where the edge node resided before the vMotion.
This would cause momentary disruption to North/South traffic until the NSX-T Edge Node VTEP MAC is learned correctly on the upstream switch/switches.
With regards to Storage vMotion, the port of NSX-T Edge Node is disconnected and enabled with vMotion flag. Hence RARP packets are sent out, too.
4 RARP packets are sent per VLAN. Hence in the worst case, with Trunk VLAN range (0-4094), 4 * 4095 = 16,380 RARP packets are expected to go out of the uplink in a period of 400ms.
Resolution
This is expected behavior in NSX-T Data Center.
Workaround: Reduce the number of Trunk VLANs to ones which are only necessary (e.g. 0-4094 to 0-1024) based on your usage. RARP Request packets during Edge Node reboot, power ON, vMotion or Storage vMotion will be reduced accordingly.