Linux virtual machines using the vmxnet3 virtual adapter lose network connectivity

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Linux virtual machines on ESX/ESXi 4.x using the vmxnet3 virtual adapter may experience a loss of network connectivity when the virtual NIC switches between offline and online.
• The /var/log/messages or dmesg file contain entries similar to:
  
  kernel: eth0: tq_error 0x80000000
  kernel: eth0: resetting
  kernel: eth0: intr type 2, mode 0, 1 vectors allocated
  kernel: eth0: NIC Link is Up 10000 Mbps

Environment

VMware ESXi 4.0.x Embedded
VMware ESXi 4.1.x Installable
VMware ESXi 4.1.x Embedded
VMware ESX 4.1.x
VMware ESXi 4.0.x Installable
VMware ESX 4.0.x

Cause

This issue is occurs due to a buffer overflow in the vmxnet3 adapter that corrupts the descriptor bit, which stalls transmit.

Resolution

This issue is resolved in:

• ESX 4.0 Patch ESX400-201203401-SG. For more information, see VMware ESX 4.0, Patch ESX400-201203401-SG: Updates VMkernel, VMX, hostd, VMware Tools (2011769).
  
• ESX/ESXi 4.1 Update 2, available at VMware Downloads.

To work around this issue, perform one of these options:

• Disable TCP Segmentation Offload (TSO)
  
  
  Note: TSO is used to reduce CPU overhead on TCP/IP. Disabling TSO may cause higher CPU during high network traffic.
  
  To disable TSO:
  1. Open an SSH or vSphere console session to the Linux virtual machine.
  2. Log in to the virtual machine.
  3. To determine the device name of your virtual network card, run the command:
     
     ifconfig
     
     
  4. To determine your current TSO setting for that adapter, run the command:
     
     ethtool -k eth1
     
     Where eth1 is the vmxnet3 adapter based on ifconfig output from step c.
     
     
  5. To disable TSO, run the command:
     
     ethtool -K eth1 tso off
     
     
• Configure the guest operating system to use a different adapter
  
  To configure the guest operating system to use a different adapter:
  1. Connect vSphere Client directly to the ESX/ESXi host or to the vCenter Server.
  2. Right-click the virtual machine and click Edit Settings.
  3. Click the network adapter and click Remove.
  4. Click OK.
  5. Right-click the virtual machine and click Edit settings.
  6. Click Add.
  7. Click Ethernet Adapter.
  8. Click Next.
  9. In Adapter Type, choose e1000 or vmxnet2 (Enhanced).
  10. If applicable for your environment, select the appropriate Network Connection.
  11. In Device Status, click Connect at power on.
  12. Click Next.
  13. Review the settings and click Finish.

Additional Information

Using Tech Support Mode in ESXi 4.1, ESXi 5.x, and ESXi 6.x
Connecting to an ESX host using an SSH client
VMware ESX 4.0, Patch ESX400-201203401-SG: Updates VMkernel, VMX, hostd, VMware Tools