Dell and Emulex iSCSI over DCB solution for VMware vSphere (Partner Verified and Supported)

Products

VMware vSphere ESXi

Issue/Introduction

This article provides information about Partner Verified and Supported Products (PVSP) for the iSCSI over DCB Solution for ESX from partners Dell and Emulex. The iSCSI over DCB solution for ESX consists of Dell EqualLogic storage arrays supporting 10GbE and Emulex OneConnect OCe11102 Converged Network Adapters (CNAs).

Note: The Partner Verified and Supported Products (PVSP) policy implies that the solution is not directly supported by VMware. For issues with this configuration, contact Dell directly. See the Support Workflow on how partners can engage with VMware. It is the partner's responsibility to verify that the configuration functions with future vSphere major and minor releases, as VMware does not guarantee that compatibility with future releases is maintained.

Disclaimer: The partner products referenced in this article are hardware devices that are developed and supported by stated partners. Use of these products are also governed by the end user license agreements of the partners. You must obtain the application, support, and licensing for using these products from the partners. For more information, see Support Information in this article.

Environment

VMware ESX 4.1.x

Resolution

Overview of Products

Dell EqualLogic storage arrays connect to the SAN over iSCSI and serve as targets. The Emulex OCe11102 CNAs are installed in servers and serve as initiators. Both products support iSCSI over DCB. Combined with a DCB enabled switch, they provide the basis for the iSCSI over DCB model for ESX.

Data Center Bridging (DCB) technology provides a set of benefits to many different enterprise applications.

iSCSI supports 10GbE today, and while it does not require DCB for it to function over Ethernet, the same benefits apply to iSCSI as apply to FCoE. iSCSI over DCB provides a lossless environment that makes TCP/IP more efficient, since it eliminates packet retransmits. iSCSI has a built in mechanism to recover from dropped packets with TCP/IP, but eliminating the need for retransmits makes the host, network, and target more efficient and reduce CPU utilization. For example:

iSCSI over DCB benefits customers running VMware since it allows for more deterministic performance in the event of network congestion, and more headroom on the host to run more virtual machines. Typical overhead of TCP/IP is about 12% of the kernel resources, which is really a fraction of overall resources. Reducing this helps overall host and target performance. iSCSI over DCB also benefits customers by providing easy, centralized configuration of network wide parameters for iSCSI, eliminating support calls for poor iSCSI performance.

The set of enhanced capabilities offered by this Dell / Emulex solution move 10GbE iSCSI from a bandwidth bump (from 1Gbe) to a truly robust iSCSI solution, ready for Enterprise SAN deployment.

This is a diagram of where the elements fit in an overall system. For illustrative purposes, these (simplified) steps occur during a read from storage:

The guest operating system requests some data from storage. This in turn translates to the Emulex adapter allocating the appropriate amount of buffer to accommodate the read, as well as issuing the read command to the EqualLogic storage target.
When the storage target receives the command, it sends data back to the adapter via the DCB-enabled switch.
The adapter places the received data into the dedicated iSCSI buffer pool.
The read data is passed back up to the guest OS requesting the information.

The implementation of iSCSI over DCB is all in hardware. The Emulex adapter presents the familiar SCSI path to the operating system.

Installation Details

All DCB support is self-contained in the Emulex iSCSI hardware adapter and EqualLogic iSCSI array (as well as the network switch connecting the two end points).

Configuration Details

Emulex OCe11000 series Adapters

Emulex has supported Priority Flow Control (802.1Qbb) and DCBX (802.1Qaz) with VMware since the introduction of the Emulex LP21000 CNA over three years ago, and more recently with the OCe10000 and OCe11000 family of Universal Converged Network Adapters. Earlier models solely supported FCoE with these protocols, the OCe11000 adds hardware support for NIC and iSCSI traffic.
As with FCoE support, the presence of DCBX and Priority Flow Control is agnostic to the host with ISCSI over DCB. These Link Layer protocols are typically enabled from the network, to ensure most efficient network configuration. No ESX APIs are used in the support of PFC or DCBX – it is all resident in the adapter hardware and control plane firmware. Upon initialization, the iSCSI firmware uses DCBX to send an application TLV to the network (known as the iSCSI TLV), and in return the switch sends the appropriate DCBX TLVs to enable Priority Flow Control, and any other parameters necessary, to complete the handshake and establish the DCB capable channel.
The Emulex iSCSI implementation is a standard ESX 4.1 / ESXi 5.0 iSCSI implementation, supporting the VMware iSCSI IMA plugin, and has IOVP certification in place.
Monitoring of this configuration is available through Emulex element management tools for the adapter, including an integrated vCenter Server plug-in (the Emulex OneCommand Mgr for vCenter).

Dell EqualLogic storage arrays

All 10Gb EqualLogic arrays shipping since Q2 2010 support iSCSI over DCB in hardware. By default, DCB is turned on and customers need only to configure the VLAN ID for DCB.
Latest MIB definitions for 802.1Qau, 802.1Qbb, 802.1Qaz.

Verification Details

Over the last three years both Emulex and Dell have attended numerous industry Plugfests sponsored by the Ethernet Alliances’ Data Center Bridging Consortium, the UNH iSCSI and DCB consortium Plugfests, as well as the UNH IOL Interoperability Integrators test lab.

Emulex has completed its DVT (Design Verification Test) suite for iSCSI over DCB for product release. The test cases include:

Entity Name	Entity Status
CNA iSCSI Management-Port Tab-DCB-GUI Local-PFC Priority Map	Passed
CNA iSCSI Management-Port Tab-DCB-GUI Local-Priority Groups-Bandwidth	Passed
CNA iSCSI Management-Port Tab-DCB-GUI InBand-PFC Priority Map	Passed
CNA iSCSI Management-Port Tab-DCB-GUI InBand-Priority Groups-Bandwidth	Passed
Test instance in Test set	Passed
Test instance in Test set	Passed
Test instance in Test set	Passed
Test instance in Test set	Passed
Test instance in Test set	Passed
Test instance in Test set	Passed
Test instance in Test set	Passed
Test instance in Test set	Passed
Test instance in Test set	Passed

Support information

The first line of support is with Dell customer support. All EqualLogic customers are required to have support agreements for their arrays. These vary from one hour response to next day for parts. Emulex will work as support partners.

Dell product and support pages:

Emulex product and support pages:

VMware Release to Release Compatibility

Because DCB support is self-contained, resident in the iSCSI hardware of the adapter and the iSCSI array (as well as network switch), release to release compatibility with ESX is expected.