Intel icen link remains down after link partner switch reboot on ESXi

Products

• VMware vSphere ESXi

Issue/Introduction

An issue occurs where a network interface using the Intel icen driver randomly remains in a DOWN state after experiencing multiple, consecutive DOWN/UP sequences (link flapping) caused by the reboot of the link partner's physical switch.

When checking the var/run/logvobd.log, messages similar to the following are observed:

<REDACTED_TIMESTAMP>: [netCorrelator] <REDACTED_SECRETS>: [vob.net.vmnic.linkstate.up] vmnic <REDACTED_HOSTNAMES> linkstate up
<REDACTED_TIMESTAMP>: [netCorrelator] <REDACTED_SECRETS>: [vob.net.vmnic.linkstate.down] vmnic <REDACTED_HOSTNAMES> linkstate down
<REDACTED_TIMESTAMP>: [netCorrelator] <REDACTED_SECRETS>: [vob.net.vmnic.linkstate.up] vmnic <REDACTED_HOSTNAMES> linkstate up
<REDACTED_TIMESTAMP>: [netCorrelator] <REDACTED_SECRETS>: [vob.net.vmnic.linkstate.down] vmnic <REDACTED_HOSTNAMES> linkstate down

Environment

VMware ESXi 
Intel icen driver versions prior to 1.14.2.0

Cause

A software defect in the Intel icen driver versions prior to 1.14.2.0 causes the network interface port to randomly remain in a DOWN state when the NIC port is configured in autonegotiation mode and subjected to rapid DOWN/UP link sequences following a link partner physical switch reboot.

Resolution

To resolve this issue, update the Intel icen driver to version 1.14.2.0 or later. Follow KB Download and install async drivers in VMware ESXi

As a  workaround to this issue, the link state can be temporarily recovered by manually toggling (bringing down and then up) the link status of the affected vmnic.

1. Log in to the ESXi CLI as root.
2. Run the following command to manually bring down the target network interface:
   esxcli network nic down -n vmnic#
3. Run the following command to manually bring up the link:
   esxcli network nic up -n vmnic#