vCenter reports Transmit packet drops for an NSX-T edge node

Products

• VMware NSX

Issue/Introduction

• Transmit packets are reported as being dropped for the NSX-T edge node in vCenter:
  
  
• Using vsish on the ESXi host and the port for the edge node VM's interface which is reporting drops, we see packet drops:
  • Login to the ESXi host via putty/ssh as root user.
  • Run the command vsish -e get /net/portsets/DvsPortset-X/ports/<Port-Number-of-the-VM>/inputStats

        SWSEC_INPUT.xxxxxxxx <swsec-input:0x########>
                pktsStarted:602584896
                pktsPassed:602526164
                pktsDropped:58732 <<<<<<<<<<<<<<<<<<<
                no client stats maintained

        VDL2_INPUT.xxxxxxxx <vdl2-leaf-in:0x#########>
                pktsStarted:27705103
                pktsPassed:21864713
                pktsDropped:5840390 <<<<<<<<<<<<<<<<<<<<<<<
                no client stats maintained

• Using the below command, the switch security packet drops are seen for "MAC CHADDR Mismatch Count"
  • Login to the ESXi host as user root via putty/ssh.
  • Run the command esxcfg-vswitch -l to make a note of the DVS switch and the corresponding DVS port UUID of the edge interface.
    
    nsxdp-cli swsec get stats --dvport <Dvport-UUID-of-the-NSX-Edge-Port> --dvs-alias nvds

         DHCPv4 Server Block Drop Count : 0
         DHCPv6 Server Block Drop Count : 0
         DHCPv4 Client Block Drop Count : 0
         DHCPv6 Client Block Drop Count : 0
         BPDU Filter Drop Count : 0
         RA Gurad Drop Count : 0
         MAC CHADDR Mismatch Count : 58751

Environment

VMware NSX-T

Cause

There are two types of packet drops found as below.

• VDL2 - These packet drops counters are incremented due to a new inter TEP feature introduced in NSX-T 3.1. When a edge node resides on a prepared host, they ensure packet destined for the edge nodes TEP interface, when arriving in the uplink, are not sent back out to the underlay.
• SWSEC - This occurs when DHCP relay is configured, relayed packets from the edge to the third party DHCP server, have source mac set to that of the gateway, but chaddr (Client hardware address) is the MAC address of the DHCP client. 

Resolution

• For the VDL2 drops, this issue is resolved in NSX-T 3.2.2, where the counter will not mark these packets as dropped packets.
  
  • For the VDL2 drops, no action is required and correct accounting will be implemented from NSX-T 3.2.2 onwards.
• For the SWSEC drops, please disable the DHCP server block setting on the Segment Security Profile applied to the segment the edge node is attached to.
  • Login to the NSX GUI as user admin.
  • Navigate to the corresponding Segment's Security Profile and Edit.
  • Ensure the DHCP --> Server Block is Disabled as shown below.