Packet drops on ESXi when a VM is injecting a tagged (802.1q) frame.

Products

• VMware vSphere ESXi
• VMware NSX

Issue/Introduction

This issue may occur in several scenarios where the following is true:

• The packets being dropped are tagged (802.1q) by the VM (especially an NSX edge VM or a 3rd party appliance VM).
• Guest Tagging/Nested Tagging is disabled for the port where the VM is attached.
• The port group/segment where the VM is attached is not set to trunking.

Cause

When a VM is attached to a DvPG or an NSX VLAN-backed segment that is non-trunking (access) and guest/nested tagging is disabled, a tagged packet is expected to be dropped. 

Here's a pktcap-uw --trace of one such packet, you can see that the packet is dropped at VLAN_InputProcessor with Drop Reason 'VlanTag Mismatch' :

Resolution

The DvPG/segment where the VM is attached can be made a trunk so that the guest tag is passed along or nested or guest tagging can be enabled, respectively:

nsxdp-cli vswitch vlan nested set --allow -p <dvport-id> -dvs <dvswitch> Note that this is only applicable for NSX segment ports.

Example: nsxdp-cli vswitch vlan nested set --allow -p port1 -dvs nsxvswitch

nsxdp-cli vswitch vlan policy set --guest-tagging -v <> [-v <>] -p <dvport-id> -dvs <dvswitch>

Example: nsxdp-cli vswitch vlan policy set --guest-tagging -v 0 -v 4 -v 10 -v 12 -v 16  -p port0 -dvs nsxvswitch (-v is for VLAN IDs).