RARP (Reverse ARP) broadcast packets are sent to the switch which the edge uplinks connected to when NSX-T Edge Nodes are rebooted/powered ON, vMotion'ed or Storage vMotion'ed
search cancel

RARP (Reverse ARP) broadcast packets are sent to the switch which the edge uplinks connected to when NSX-T Edge Nodes are rebooted/powered ON, vMotion'ed or Storage vMotion'ed

book

Article ID: 322563

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

This article is to provide information about RARP packet behavior during VM reboot/power ON or vMotion of guest VMs belonging to a trunk VLAN

Symptoms:

  • The edge node uplink is configured with a trunk VLAN (0-4096) for connectivity to the physical switch.
  • 10,000+ RARP packets are broadcast to the physical switch via the edge node uplink interface in a period of 400 milliseconds, when performing edge reboot/power ON Or vMotion/Storage vMotion of the edge node. 
  • This leads the physical switch to detect a broadcast storm if it has a storm control configuration like "storm-control broadcast level pps 10k".



Environment

VMware NSX-T Data Center

VMware NSX-T Data Center 3.x

VMware NSX 4.x

Cause

  • RARP packets are sent to the edge nodes uplink physical switch, when during vMotion of an NSX-T Edge Node, so that physical switch can learn MAC address of NSX-T Edge Node's fp-ethX (VTEP) interface.
  • This behaviour is also during Edge Node reboot or power ON or any other operations where its vNICs are activated.
  • Without these RARP packets, packets destined to the NSX-T Edge Node VTEP interface could continue to be forwarded to the ESXi host where the edge node resided before the vMotion.
  • This would cause momentary disruption to North/South traffic until the NSX-T Edge Node VTEP MAC is learned correctly on the upstream switch/switches.
  • With regards to Storage vMotion, the port of NSX-T Edge Node is disconnected and enabled with vMotion flag. Hence RARP packets are sent out, too.
  • 4 RARP packets are sent per VLAN. Hence in the worst case, with Trunk VLAN range (0-4094), 4 * 4095 = 16,380 RARP packets are expected to go out of the uplink in a period of 400ms.

Resolution

This is expected behavior in NSX-T Data Center.

Workaround:
Reduce the number of Trunk VLANs to ones which are only necessary (e.g. 0-4094 to 0-1024) based on your usage. RARP Request packets during Edge Node reboot, power ON, vMotion or Storage vMotion will be reduced accordingly.