RDMA connections do not fail over to other RDMA-capable NICs within a NIC team

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

RDMA connections do not fail over to other RDMA-capable NICs within a NIC team. When an RDMA-capable NIC goes offline, RDMA connections fail regardless of any association with a NIC team. Services relying on RDMA connections might show failures and write errors to logs.

 

You can add NICs to a NIC team to allow traffic failover to other NICs in the team from a NIC that is offline, but the NIC teaming feature is specific to networking traffic and does not apply to RDMA traffic. This means that even though a device might be in a NIC team and might have RDMA traffic, the properties of the NIC team do not extend to the RDMA traffic on that device. As a result, even if multiple NICs in a NIC team are RDMA-capable, the NIC team relationship has no impact on RDMA traffic and connections still fail.

Environment

• VMware vCenter Server 6.0.x
• VMware vSphere ESXi 6.0

Resolution

If you are using a NIC for RDMA traffic, take the NIC offline only after the RDMA traffic has concluded. Taking the NIC offline while it is processing RDMA traffic causes traffic on this NIC to stop regardless of NIC teaming configurations.