Duplicate Multicast or Broadcast Packets are Received by a Virtual Machine When the Interface is Operating in Promiscuous Mode

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:
The following symptoms are observed:

• Promiscuous mode is set to accept on the virtual switch portgroup and the virtual machine guest OS places its vNIC in promiscuous mode
• When running a packet capture within the VM, multicast and broadcast packets are received multiple times
• The vSwitch is configured with NIC teaming and the load balancing policy is Route Based on Originating Port ID, Route based on Source MAC Hash, Use explicit failover order or Route Based on Physical NIC Load
• Multicast applications and protocols (such as CARP) running in virtual machines in promiscuous mode experience problems due to duplicated receive packets

Cause

In a NIC teamed environment where multiple uplinks are configured for a virtual switch and a port channel or LACP is not configured on the physical switch, the vSwitch will receive a multicast or broadcast packet from the physical network on each vSwitch uplink in the NIC team.  All traffic received by the vSwitch will be forwarded to the virtual portgroup in promiscuous mode so the virtual machine guest OS will receive multiple multicast or broadcast packets.

For more information on promiscuous mode, see How promiscuous mode works at the virtual switch and portgroup levels.

Resolution

To prevent this issue, you must enable the /Net/ReversePathFwdCheckPromisc setting:
 

esxcli system settings advanced set -o /Net/ReversePathFwdCheckPromisc -i 1

To check that the setting is enabled, run the following command:
 

esxcli system settings advanced list -o /Net/ReversePathFwdCheckPromisc
   Path: /Net/ReversePathFwdCheckPromisc
   Type: integer
   Int Value: 1
   Default Int Value: 0
   Min Value: 0
   Max Value: 1
   String Value:
   Default String Value:
   Valid Characters:
   Description: Block duplicate packet in a teamed environment when the virtual switch is set to Promiscuous mode.

This setting will discard packets coming from uplinks that are not associated with the particular client when promiscuous mode is enabled and will prevent duplicate packets from being received by the guest operating system.  This will affect all promiscuous mode virtual machine and vmkernel interfaces on the ESXi host.

In order for the setting to take effect, promiscuous mode must be toggled off and on by the guest operating system.  An operation such as a guest OS reboot or a vMotion to another ESXi host with the /Net/ReversePathFwdCheckPromisc setting enabled is sufficient.  The setting does not require a reboot of the ESXi host to take affect.

You can verify if /Net/ReversePathFwdCheckPromisc is enabled with the following steps:
 

1. Verify the port number for the virtual machine interface.
 

net-stats -l | grep <VM name>
67108871            5       7 DvsPortset-1     00:0c:29:82:24:cf  <VM name>.eth2

2. Check the stats of the portgroup using the port number from the output of command #1 (bolded section indicates the filter is applied):

vsish -e get /net/portsets/DvsPortset-1/ports/67108871/outputStats
io chain stats {
   starts:8749231
   resumes:0
   inserts:0
   removes:0
   errors:0
   pktsStarted:9011842
   pktsPassed:4508858
   pktsDropped:0
   pktsCloned:9011785
   pktsFiltered:4502984
   pktsFaulted:0
   pktsQueued:0
   pktErrors:0
   pktsInjected:0
   functions:
        PRE_FILTER <VLAN_OutputProcessor@(etherswitch)#<None>:0x43014f8b6e10>
                pktsStarted:9011842
                pktsPassed:9011842
                pktsDropped:0
                pktsFiltered:0
                pktsQueued:0
                pktsFaulted:0
                pktsInjected:0
                pktErrors:0

        FILTER <Team_ReverseFilterPerList@(etherswitch)#<None>:0x43014f8b6b90>
                pktsStarted:9011842
                pktsPassed:4508858
                pktsDropped:0
                pktsFiltered:4502984
                pktsQueued:0
                pktsFaulted:0
                pktsInjected:0
                pktErrors:0

        TERMINAL <<VM Name>.eth2:0x0>
                pktsStarted:4508858
                pktsPassed:0
                pktsDropped:4508858
                no client stats maintained

}

The above output shows that Team_ReverseFilter is attached to the vSwitch port and it is filtering packets.