Broadcom adapter data corruption on ESXi/ESX hosts using the tg3 driver and TSO enabled
Article ID: 301636
Updated On:
Products
VMware
Issue/Introduction
Symptoms:
On ESXi/ESX hosts where:
On ESXi/ESX hosts where:
- The system is configured with greater than 4 GB RAM
- TCP Segmentation Offload (TSO) is enabled on the adapter
- The system is running ESXi/ESX 3.5 or later
- The tg3 driver version is less than v3.135b
Cause
Broadcom NIC devices, such as the NetXtreme I with the BCM5719 chipset, support LSO/TSO in hardware. An issue in this TSO hardware engine can cause data corruption when:
- TSO is enabled
- A TSO operation is being performed on TCP Payload data and the address of the host physical memory buffer of any of the segments corresponding to this crosses a 4 GB address threshold.
Resolution
This is a known issue affecting tg3 driver versions earlier than 3.135b.
To resolve this issue on ESXi 5.x, update the tg3 driver:
You can download the latest driver from the VMware downloads page.
For ESXi/ESX 3.5/4.x, there is currently no persistent fix for the Broadcom tg3 driver. The only available workaround is to disable the TSO feature after the server boots.
For more information on disabling TSO, see Understanding TCP Segmentation Offload (TSO) and Large Receive Offload (LRO) in a VMware environment (2055140).
Note: Some versions of the tg3 drivers have the TSO feature disabled by default and are not exposed to this issue, unless you manually enabled the TSO feature in the driver. For more information on default TSO settings for the various driver versions, see the tables below.
All driver versions 3.135b and later will not experience this issue, regardless of whether TSO is enabled or disabled.
IOVP tg3 drivers:
To resolve this issue on ESXi 5.x, update the tg3 driver:
- ESXi 5.5: version 3.135b.v55.1 (released on 12/27/2013) or later
- ESXi 5.0 / 5.1: version 3.135b.v50.1 (released on 12/27/2013) or later
You can download the latest driver from the VMware downloads page.
Workaround
If you are unable to update the tg3 driver, you can work around this issue by disabling the TSO feature in the driver.For ESXi/ESX 3.5/4.x, there is currently no persistent fix for the Broadcom tg3 driver. The only available workaround is to disable the TSO feature after the server boots.
For more information on disabling TSO, see Understanding TCP Segmentation Offload (TSO) and Large Receive Offload (LRO) in a VMware environment (2055140).
Note: Some versions of the tg3 drivers have the TSO feature disabled by default and are not exposed to this issue, unless you manually enabled the TSO feature in the driver. For more information on default TSO settings for the various driver versions, see the tables below.
All driver versions 3.135b and later will not experience this issue, regardless of whether TSO is enabled or disabled.
TSO default settings
Inbox tg3 drivers:| ESX/ESXi version | ||||||||
| 3.5 GA / U1 | 3.5 U2 / U3 U4 | 3.5 U5 | 4.0 GA / U1 U2 / U3 | 4.1 GA / U1 U2 / U3 | 5.0 GA / U1 U2 / U3 | 5.1 GA / U1 U2 | 5.5 GA | |
TCPv4 TSO default setting | Disabled | Enabled | Disabled | Disabled | Disabled | Disabled | Disabled | Enabled |
Driver version | 3.43b | 3.81c | 3.81c | 3.86 | 3.86.0.1-1vmw 3.110h.v41.1 | 3.110h.v50.4 3.123b.v50.1 | 3.110h.v50.4 | 3.123c.v55.5-1vmw |
IOVP tg3 drivers:
| ESX/ESXi version | ||||||
| 3.5 GA / U1 U2 / U3 U4 / U5 | 4.0 GA / U1 U2 / U3 / U4 | 4.1 GA / U1 U2 / U3 | 5.0 GA / U1 U2 / U3 | 5.1 GA / U1 U2 | 5.5 GA | |
TCPv4 TSO default setting | Enabled | Enabled | Enabled | Enabled | Enabled | Enabled |
Driver version | 3.120h.v40.1 3.123b.v40.1 3.124c.v40.1 3.129d.v40.1 | 3.120h.v40.1 3.123b.v40.1 3.124c.v40.1 3.129d.v40.1 | 3.120h.v50.2 3.123b.v50.1 3.124c.v50.1 3.129d.v50.1 3.131d.v50.1 3.133d.v50.1 3.133d.v55.1 3.134e.v50.1 | 3.123b.v50.1 3.124c.v50.1 3.129d.v50.1 3.131d.v50.1 3.133d.v50.1 3.133d.v55.1 3.134e.v50.1 | 3.123c.v55.5 3.133d.v55.1 3.134e.v55.1 | |
Additional Information
- To determine if TSO is enabled, see Determining if hardware TSO is enabled on the host in Understanding TCP Segmentation Offload (TSO) and Large Receive Offload (LRO) in a VMware environment (2055140).
- To check the tg3 driver version on ESXi 5.x hosts, run this command:
vmkload_mod -s tg3 | grep Version - For more information on checking NIC drivers and firmware, see Determining Network/Storage firmware and driver version in ESXi/ESX 4.x and ESXi 5.x (1027206).
To be alerted when this document is updated, click the Subscribe to Article link in the Actions boxDetermining Network/Storage firmware and driver version in ESXi 4.x and later
Understanding TCP Segmentation Offload (TSO) and Large Receive Offload (LRO) in a VMware environment
tg3 ドライバを使用した TSO が有効な ESXi/ESX ホストでの Broadcom アダプタのデータ破損
在使用 tg3 驱动程序并启用 TSO 的 ESXi/ESX 主机上 Broadcom 适配器数据损坏
Feedback
Yes
No
Powered by
