Hosts May Disconnect from vCenter when using a X710 or XL710 or X722 family card when LLDP is enabled.
Article ID: 324503
Updated On:
Products
Issue/Introduction
Why This Happens:
By design, vmk0 copies the physical MAC address of vmnic0. This is useful for some customers as they can prepare DHCP reservations ahead of the hardware arriving as server vendors will give you the MAC address for the onboard adapters once it is assembled. This family of network adapters has an embedded Link Layer Discovery Protocol (LLDP) engine that will keep issuing ARP requests of that MAC to the first port even if VMK0 moves to another port. This can cause the host to disconnect from vCenter if vmk0 is used for management traffic as the CAM table on the switch above gets confused about where to send packets.
Example:
If vmnic0 has a mac of 00:00:00:00:00:00 and vmnic1 has a mac of 11:11:11:11:11:11. The vmkernel adapter vmk0 will have mac 00:00:00:00:00:00. If LLDP packets are coming from vmnic0 (physical) over layer 2 to 00:00:00:00:00:00 and vmk0 moves to vmnic1, the physical switch CAM tables won’t know how to get the traffic to the correct ethernet port reliably because vSphere's attempt to issue a reverse ARP will conflict with LLDP's request.
Troubleshooting:
If you have syslog configured for your physical switches you may see Mac Flapping (MACFLAP on Cisco) entries in the switch logs. This may also manifest itself in the switch logs as a high volume of CAM table updates as the 2 ports fight over the MAC Address.
vobd.log will report mac conflicts using the events "VmMacConflictEvent" or "VmStaticMacConflictEvent"
Environment
Resolution
There is no solution, however using any of the following methods can help to work around the issue.
- Create, but don’t use VMK0 for anything.
- Put VMK0 on a tagged NON-Native VLAN. This will prevent conflict with the LLDP agent and MAC address if it's on a completely different broadcast domain.
- Run LACP across the original physical port and another port.
- Disable the NIC's built-in LLDP engine (see below).
Network driver i40en version 1.5.6 introduces additional parameters that allow you to allow or disallow the flow of LLDP frames to the network stack. This is currently supported with Intel X710/XL710/XXV710 adapters using FW 6.0.x and later and Intel X722 adapters using FW 4.0 or later.
Download: https://customerconnect.vmware.com/web/vmware/details?downloadGroup=DT-ESXI65-INTEL-I40EN-156&productId=614
Note: You may also have to change the "LLDP Agent" setting to "Disabled" in the NIC BIOS for each port. Please contact your hardware vendor for exact instructions to accomplish this. The hardware vendor will also know the correct commands to run to disable or enable LLDP on the physical NICs.
For 2-port NICs:
To Disable LLDP: esxcli system module parameters set -m i40en -p LLDP=0,0
To Enable LLDP: esxcli system module parameters set -m i40en -p LLDP=1,1
For 4-port NICs:
To Disable LLDP: esxcli system module parameters set -m i40en -p LLDP=0,0,0,0
To Enable LLDP: esxcli system module parameters set -m i40en -p LLDP=1,1,1,1
Note: A reboot of the host is required for the changes to take effect.
For more information on resolving MAC address conflicts, see vmk0 management network MAC address is not updated when NIC card is replaced or vmkernel has duplicate MAC address
Additional Information
Feedback
