Esxi may lose network management when using a Broadcom NetXtreme-E network card

Products

• VMware vSphere ESXi

Issue/Introduction

This article describes the troubleshooting of management network connectivity after  failover when  using Broadcom NetXtreme-E network card.
After network failover (vmnic down or up), esxi management network becomes unreachable and  may be disconnected from vCenter server

On physical switches, you can see vmk0 Mac Address (= vmnic0 mac address) on 2 locations
You can also see other NetXtreme-E network cards mac addresses (not only vmnic0) on physical switch.

Environment

• vSphere ESXi 7.x
• vSphere ESXi 8.x

Cause

By design, vmk0 copies the physical MAC address of vmnic0. This is useful for some customers as they can prepare DHCP reservations ahead of the hardware arriving as server vendors will give you the MAC address for the onboard adapters once it is assembled. This family of network adapters has an embedded Link Layer Discovery Protocol (LLDP) engine that will keep issuing ARP requests of that MAC to the first port even if VMK0 moves to another port. This can cause the host to disconnect from vCenter if vmk0 is used for management traffic as the CAM table on the switch above gets confused about where to send packets. 

Resolution

Use on of the following solution

1/ Disable LLDP nearest bridge in BIOS Device Configuration Menu
for example
https://techdocs.broadcom.com/us/en/storage-and-ethernet-connectivity/ethernet-nic-controllers/bcm957xxx/adapters/installation/uefi-hii-menu.html

https://dl.dell.com/manuals/all-products/esuprt_data_center_infra_int/esuprt_data_center_infra_network_adapters/broadcom-netxtreme-adapters_users-guide4_en-us.pdf

2/ Create, but don’t use VMK0 for anything

Additional Information

Impact/Risks:
esxi management network becomes unreachable for a short period of time