RDMA adapter is not visible under physical adapters after installing RDMA driver

Products

• VMware vSphere ESXi

Issue/Introduction

• After installing the RDMA driver on an ESXi host, the RDMA adapter is not visible under Networking ,Physical adapters
  
  
• When running the command: esxcli rdma device list, no devices are listed

Environment

• ESXi 8.x
• ESX 9.x

Cause

Installing the drivers is not enough to enable RDMA on the adapter. The driver module must explicitly be initialise to switch on RDMA functionality.

Resolution

NOTE: Refer to vendor-specific documentation for precise configuration instructions and also Configuring RDMA for vSAN

Below is an example on how to enable RDMA for Intel Adapter on the ESXi host:

Step 1: SSH to ESXi Host

• Connect to the ESXi host using SSH with root credentials.

Step 2: Check if RDMA Driver is Loaded

• Run
esxcli rdma device list


  If no devices are listed, the RDMA driver might not be loaded
  
  

Step 3: Verify and Load RDMA Driver Module

• List loaded kernel modules:

  vmkload_mod -l | grep -i irdma

• If not loaded, manually load the RDMA driver:

  vmkload_mod irdma

• Confirm it loaded:

  vmkload_mod -s irdma



Step 4: Enable RDMA and RoCE on the Adapters

    For Intel NICs (icen driver):

• Enable RDMA on ports 0 and 1:

  esxcli system module parameters set -m icen -p "RDMA=1,1"

• Enable RoCE on irdman module:

  esxcli system module parameters set -m irdman -p "ROCE=1,1"

Details:

• icen → Intel Ethernet devices (e.g., E810 series).

• RDMA=1,1 → Enables RDMA on NIC port0 and port1.

• irdman → Intel RDMA management driver.

• ROCE=1,1 → Enables RoCE (RDMA over Converged Ethernet) on two interfaces.

 For ESXi Hosts with 4 RDMA NICs:

• Enable RDMA and RoCE on all four ports:

   

  esxcli system module parameters set -m icen -p "RDMA=1,1,1,1"

  esxcli system module parameters set -m irdman -p "ROCE=1,1,1,1"

Note:
Each number represents a physical port (port0, port1, port2, port3).
1 = enable; 0 = disable.

To Verify Current Settings:

esxcli system module parameters list -m icen

esxcli system module parameters list -m irdman

Step 5: Apply Specific Port Configurations

Scenario	Command
Enable RDMA/RoCE on vmnic0 & vmnic1; disable on vmnic2 & vmnic3	esxcli system module parameters set -m icen -p "RDMA=1,1,0,0" esxcli system module parameters set -m irdman -p "ROCE=1,1,0,0"
Enable RDMA/RoCE on vmnic2 & vmnic3; disable on vmnic0 & vmnic1	esxcli system module parameters set -m icen -p "RDMA=0,0,1,1" esxcli system module parameters set -m irdman -p "ROCE=0,0,1,1"
Disable RDMA/RoCE on all ports	esxcli system module parameters set -m icen -p "RDMA=0,0,0,0" esxcli system module parameters set -m irdman -p "ROCE=0,0,0,0"

 

Step 6: Reboot the Host

After applying the configuration changes, you must reboot the ESXi host for the settings to take effect.

Additional Information

• If RDMA driver are installed and the modules are loaded the RDMA driver will be visible under ESXi-->Configure--->Networking tab

• For Broadcom RDMA Nics, perform the below steps to enable rdma

  esxcli system module parameters set -m bnxtnet -p disable_roce=0
esxcli system module parameters set -m bnxtroce -p disable_rocev2=0  (For ROCEv2 mode)
  
  Notes:
  ESX host reboot is required after enabling the modules.
  Ensure in the BIOS of the host, on the Device Level Configuration page, (NIC+RDMA mode) is Enabled for the RDMA NICs.