A physical NIC may be brought down due to frequent link status flapping

Products

• VMware vSphere ESXi

Issue/Introduction

• When a physical NIC experiences frequent link status flapping (i.e. consecutive link down and link up events), ESXi may turn off the physical NIC to avoid unnecessary link failover and network instability.
• For example, a physical NIC driver may reset the physical NIC when it detects "tx Hang". A reset causes a link down/up event to be generated. If the physical NIC is itself bad, this causes periodic reset and link status flapping. The vmkernel.log log file on the ESXi host will show that one or more network cards and repeated state change between "Up" and "Down".
• During link status flapping, traffic may migrate between different physical NICs. If the flapping is too frequent, you experience network issues, such as bandwidth reduction, an unstable network or ESX Host lost Network Connectivity.
• Similar logs can be noticed in vobd.log
  

  In(14) vobd[2098027]:  [netCorrelator] 764747381136us: [esx.problem.net.vmnic.linkstate.flapping] Taking down physical NIC vmnic5 because the link is flapping.
  In(14) vobd[2098027]:  [netCorrelator] 764820363271us: [esx.problem.net.vmnic.linkstate.flapping] Taking down physical NIC vmnic5 because the link is flapping.

• hostd.log reports below events
  

  In(166) Hostd[2103930]: [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 132032 : Taking down physical NIC vmnic5 because the link is unstable.
  In(166) Hostd[2103925]: [Originator@6876 sub=Vimsvc.ha-eventmgr] Event 132175 : Taking down physical NIC vmnic5 because the link is unstable.

• Checking nic status with "esxcli network nic get -n vmnic#" command shows "Link Status: Down by linkflapping".
  
  

Cause

This issue occurs when the ESXi host detects frequent link status changes (flapping) on a physical Network Interface Card (NIC). To prevent network instability and unnecessary link failover events, the ESXi host automatically disables the unstable NIC.

This protective mechanism is triggered when the number of link status changes exceeds the configured threshold in the Net.LinkFlappingThreshold parameter (default is 60 events per minute).

Common causes for link flapping include:

• Hardware faults within the physical NIC.

• The NIC driver resetting the device after detecting errors, such as a "tx Hang".

• Physical layer issues, such as faulty cabling or switch port errors.

Resolution

ESX automatically alerts you to the bad hardware and disables the device to avoid the link flapping state. Work with hardware vendor or physical network team to validate the link/switchport to isolate and fix the flapping issue.

• If the cause of link flapping is a hardware fault and the link does not come back up even after repair, please run the following command:
• esxcli network nic up -n vmnicX

Additional Information

This behavior could be adjusted by an ESX parameter "Net.LinkFlappingThreshold". Refer to The ESXi parameter "LinkFlappingThreshold" to adjust the link flap protection mechanism

• By default, the Net.LinkFlappingThreshold is set to 60, so if link flapping occurs more than 60 times per minute, the NIC will become unusable.

Network adapter (vmnic) is down or fails with a Failed Criteria Code