RDMA and iSER adapters are missing after upgrade of ESXi host with Marvell qedentv inbox driver to a custom image using qedentv async driver

Products

• VMware vSphere ESXi

Issue/Introduction

After upgrading an ESXi host from a build using the Marvell E4 inbox driver (qedentv) to a custom image (such as HPE Custom Image) containing the qedentv async driver, users will encounter the following symptoms:

• qedrntv RDMA devices (vmrdma) are no longer present in the ESX system.
• iSER (iSCSI Extensions for RDMA) adapters and setup disappear and the connections to iSER targets are lost.

Environment

VMware ESXi with Marvell E4 inbox driver and RDMA/iSER configured.

Cause

Marvell E4 qedentv driver has a driver module parameter "enable_roce".
In qedentv inbox driver, the default value of "enable_roce" is 1, but in qedentv async driver, the default value of "enable_roce" is 0.

After upgrading from ESX with inbox driver to ESX with qedentv async driver, the value of "enable_roce" will be re-set to 0, and the RDMA/iSER setup will disappear since RoCE is no longer enabled in qedentv async driver.  

Resolution

To prevent occurrence of this issue, the value must be set before the upgrade:

1. Put the host into maintenance mode.
2. SSH to the ESXi host.
3. Set the parameter on qedentv driver:
   # esxcli system module parameters set -m qedentv -p 'enable_roce=1'
4. Reboot the ESXi host.
5. If needed, you can run the command below to confirm the driver module parameter has been set:
   # esxcli system module parameters list -m qedentv | grep enable_roce