IPV6 ND packets or Multicast Packets with Dest MAC as multicast MAC is dropped at the receiver VNIC

Products

• VMware NSX

Issue/Introduction

Symptoms:
IPV6 Neighbor Discovery packets or Multicast Packets with destination MAC as multicast MAC is dropped at the receiver VNIC.
 

Using the drop filter, check if the packet is dropped on the switch port corresponding to the destination VM.

pktcap-uw --switchport <switchport> --capture DROP

Run the below command on the ESX host where the problem VM is located.

Non-Working VM:
 

• net-stats -l
• From the output of above command, get the portNum and switchName values that correlate with your VM, and substitute their values in the below command
  vsish
  cat /net/portsets/<switchName>/ports/<portNum>/status
  Example:
  cat /net/portsets/DvsPortset-1/ports/100663504/status
  port {
  port index:208
  vnic index:0x00000001
  portCfg:c468b258-####-####-####-########ab5
  dvPortId:b9fc34a5-####-####-####-########f74
  clientName:client.eth1
  clientType: 5 -> VMM Virtual NIC
  clientSubType: 9 -> Vmxnet3 Client

world leader:5803472

flags: 0x40093 -> IN_USE ENABLED WORLD_ASSOC DVS_PORT CONNECTED

Impl customized blocked flags:0x00000000

Passthru status: 0x8 -> DISABLED_BY_HOST

fixed Hw Id:##:##:##:##:##:45

ethFRP:frame routing {

requested:filter {

flags:0x0000000b -----------------> The "0x0000000b" doesn't have VMK_ETH_FILTER_ALLMULTI flag.

unicastAddr:##:##:##:##:##:45:

numMulticastAddresses:4

multicastAddresses:##:##:##:##:##:01,##:##:##:##:##:##,33:##:ff:35:02:12,33:##:ff:b7:##:45,

LADRF:[0]: 0x0

       [1]: 0x0

}

accepted:filter

flags:0x0000000b

unicastAddr:##:##:##:##:##:45:

numMulticastAddresses:4

multicastAddresses:##:##:##:##:##:01,##:##:##:##:##:##,33:33:##:35:02:12,33:##:ff:b7:##:45,

LADRF:[0]: 0x0

         [1]: 0x0
      }

}

For working VM:

   ethFRP:frame routing {

      requested:filter {

         flags:0x0000000d ----------------------->  The "0x0000000d" has VMK_ETH_FILTER_ALLMULTI flag.

         unicastAddr:##:##:##:##:##:c4:

         numMulticastAddresses:0

 

Environment

VMware NSX-T Data Center
VMware NSX-T Data Center 3.x

Cause

It depends on the number of multicast addresses enabled. If the total is less than 32, perform perfect filtering based on exact multicast address.

Otherwise, enable the allmulti flag so that all multicast address is received.

 

There are two main ways in which this ALLMULTI flag can be set.

    (3.1) When multicast snooping is enabled, and the number of multicast addresses on a port > 512, ALLMULTI is set. 

    (3.2) ALLMULTI is set from the VM by vmxnet3 Linux driver.  Looking at vmxnet3 Linux driver code, this happens in two cases:

          (3.2.1) Explicit request to vmxnet3 driver to set ALLMULTI.

          (3.2.2) If somehow the driver's attempt to pass an exact list of multicast addresses to the vmxnet3 device fails, ALLMULTI is set instead.  When this happens, the following log entry should be found in dmesg when this happens: "failed to copy mcast list, setting ALL_MULTI"

Resolution

Workaround:

• Add Static IPV6 ND entries, but it has to be done for every device that needs to reach the VM

(OR)

• For workaround, enable all multi on the problem Vm using the below command

"ifconfig eth1 allmulti" on issue vms 

Before setting allmulti on VM flags, it might be

 

eth1: flags <UP,BROADCAST,RUNNING,MULTICAST>

and after setting that, it might be 

eth1: flags <UP, BROADCAST,RUNNING,ALLMUTICAST,MULTICAST>. 

Then check whether the switchport status of vsish node is set as 

   ethFRP:frame routing {

      requested:filter {

      flags:0x0000000d