VLSR - Unable to discover NVME storage on one vmhba device. An error occurred during host configuration: Unable to connect to the specified transport target using the provided connection specification.

Products

• VMware vSphere ESXi

Issue/Introduction

Unable to discover NVME storage on one vmhba device.  Host reports error: An error occurred during host configuration: . Unable to connect to the specified transport target using the provided connection specification.


VMkernel.log:

2024-12-16T17:06:16.383Z Al(177) vmkalert: cpu44:8749658)ALERT: nvmerdma: 465: No tagged vmknic interface found. Please tag relevant vmknic(s) for steering NVMe/RDMA traffic correctly.
2024-12-16T17:06:16.383Z In(182) vmkernel: cpu44:8749658)nvmerdma:476 [ctlr 329, queue 0] Failed to get source ip address for uplink vmnic10: Not found

2024-12-16T00:21:33.834Z Wa(180) vmkwarning: cpu9:8697676)WARNING: NVMFDEV:3554 vmhba66 adapter find in the global connection list failed: Not found

Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

Since one vmhba is working and the other isn't, it suggests that the issue might be specific to the non-working VMHBA.

This could be due to a configuration problem on the virtual switch, missing VLAN tags, or a configuration problem on the fabric at the host level.

Resolution

Check VMkernel NIC Tagging:

Ensure that the VMkernel NIC is tagged correctly for NVMe/RDMA traffic. You can use the following command to tag the NIC:

esxcli network ip interface tag add --interface-name vmkN --tagname NVMeTCP
 
Replace vmkN with the appropriate VMkernel NIC ID.

Verify RDMA Configuration:

Ensure that the RDMA configuration is correct and that the network adapter supports RDMA. You can check this with:

esxcli network nic list

Look for the RDMA capability in the output.

 

Try to perform a rescan operation to see if it gets detected:

esxcli storage core adapter rescan 

esxcli storage core adapter list

Then try to reset the HBA from command line:

esxcli storage san fc reset -A vmhba66

 

Verify Fabric Configuration on Host:

Check the configuration of the uplinks on the fabric at the host level.

In this example, we found that the uplink for Fabric B on the host was not correctly configured and was missing its required VLAN.