Post the patch upgrade the ESXi host, the shared NVMe devices are no longer visible
search cancel

Post the patch upgrade the ESXi host, the shared NVMe devices are no longer visible

book

Article ID: 313961

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

This article aims to offer direction and steps to troubleshoot issues concerning the connectivity of NVMe storage between an ESXi host and storage

Upon patch upgrading the ESXi host, the NVMe namespaces and datastores that were previously visible are not appearing in the vSphere Client.

From vmkernel.log we see the below events.

[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)NVMFDEV:177 vmkParams.hostid:  ########-####-####-####-########6410
[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)NVMFDEV:178 vmkParams.hostnqn: nqn.2014-08.in.xxxx.####.####:nvme:####-ESX01
[YYYY-MM-DDTHH:MM:SS] cpu1:2098192)WARNING: NVMFDEV:882 Failed to transport connect controller 257, status: Failure
[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)NVMFDEV:179 vmkParams.subnqn:  nqn.1988-11.com.dell:powerstore:00:###########
[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)NVMFDEV:190 vmkParams.trType:  FC
[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)NVMFDEV:192 vmkParams.trwwnn:  ############
[YYYY-MM-DDTHH:MM:SS] cpu1:2098192)NVMEDEV:1565 Ctlr 257 freeing
[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)NVMFDEV:194 vmkParams.trwwpn:  ############
[YYYY-MM-DDTHH:MM:SS] cpu1:2098192)NVMEDEV:9067 Cancel requests of controller 257, 0 left.
[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)WARNING: lpfc: lpfc_remoteport_add:1875: 0:6067 No match ############
[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)WARNING: lpfc: lpfcGetRemotePort:2491: 0:1073 Get Remote Port Failed
[YYYY-MM-DDTHH:MM:SS] cpu26:2098194)NVMEDEV:1393 Ctlr 259 allocated, maximum queues 16
[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)WARNING: lpfc: lpfcConnectCtrlr:2543: 0:1076 Failed in Remote Port Get WWNN : ########### WWPN : ###########
[YYYY-MM-DDTHH:MM:SS] cpu12:2098193)WARNING: NVMFDEV:882 Failed to transport connect controller 258, status: Failure
[YYYY-MM-DDTHH:MM:SS] cpu1:2098192)WARNING: NVMFEVT:1084 Failed to connect controller nqn.1988-11.com.####:########:00:#############, status: Failure 

From Boot.gz the below event appear in logs .

boot:2026-01-28T11:29:48.652Z cpu0:2097887)NVMFDEV:172 target type:NVMe
boot:2026-01-28T11:29:48.652Z cpu15:2097886)NVMFDEV:172 target type:NVMe
boot:2026-01-28T11:29:48.652Z cpu0:2097887)NVMFDEV:180 vmkParams.asqsize:32
boot:2026-01-28T11:29:48.652Z cpu15:2097886)NVMFDEV:180 vmkParams.asqsize:32
boot:2026-01-28T11:29:48.652Z cpu0:2097887)NVMFDEV:181 vmkParams.cntlid:0xffff
boot:2026-01-28T11:29:48.652Z cpu15:2097886)NVMFDEV:181 vmkParams.cntlid:0xffff
boot:2026-01-28T11:29:48.652Z cpu0:2097887)NVMFDEV:182 vmkParams.hostid:6#######-####-####-####-############
boot:2026-01-28T11:29:48.652Z cpu0:2097887)NVMFDEV:184 vmkParams.hostnqn:nqn.2014-08.com.test abc:nvme:##########
boot:2026-01-28T11:29:48.652Z cpu0:2097887)NVMFDEV:185 vmkParams.subnqn:nqn.1988-11.com.dell:powerstore:##:####################
boot:2026-01-28T11:29:48.652Z cpu26:2097885)WARNING: NVMFDEV:1124 Failed to transport connect controller 275, status: Failure
boot:2026-01-28T11:29:48.652Z cpu15:2097886)NVMFDEV:184 vmkParams.hostnqn:nqn.2014-08.com.test abc:nvme:##########
boot:2026-01-28T11:29:48.652Z cpu0:2097887)NVMFDEV:196 vmkParams.trType:FC
boot:2026-01-28T11:29:48.652Z cpu15:2097886)NVMFDEV:185 vmkParams.subnqn:nqn.####-##.com.test:powerstore:##:####################
boot:2026-01-28T11:29:48.652Z cpu0:2097887)NVMFDEV:197 vmkParams.trwwnn:58##############
boot:2026-01-28T11:29:48.652Z cpu15:2097886)NVMFDEV:196 vmkParams.trType:FC
boot:2026-01-28T11:29:48.652Z cpu0:2097887)NVMFDEV:199 vmkParams.trwwpn:58##############
boot:2026-01-28T11:29:48.652Z cpu15:2097886)NVMFDEV:197 vmkParams.trwwnn:58##############
boot:2026-01-28T11:29:48.652Z cpu15:2097886)NVMFDEV:199 vmkParams.trwwpn:58##############
boot:2026-01-28T11:29:48.652Z cpu31:2097888)NVMFDEV:164 Controller 278,connecting using parameters:
boot:2026-01-28T11:29:48.652Z cpu31:2097888)NVMFDEV:167 kato:10000
boot:2026-01-28T11:29:48.652Z cpu31:2097888)NVMFDEV:168 subtype:2
boot:2026-01-28T11:29:48.652Z cpu31:2097888)NVMFDEV:169 cdc:0

Environment

VMware vSphere ESXi 6.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

Post the ESXi host patch upgrade, there was an alteration in the host nqn, which is derived from the fqdn. In this instance, the host fqdn is “ABC-DEF-ESX01.domain.corp.co.in”.

Before the upgrade, the host nqn was “nqn.2014-08.com.vmware:nvme:xxxxxx”. However, following the upgrade, the host nqn transitioned to “nqn.2014-08.in.co.corp.domain:nvme:ABC-DEF-ESX01”.

Due to the lack of updates in the initiator section of the storage console reflecting these changes, the ESXi host was unable to establish a session with the storage.

Resolution

Update the initiators in the storage console with the latest host nqn i.e. from  “nqn.2014-08.com.vmware:nvme:xxxxxx” to latest “nqn.2014-08.in.co.corp.domain:nvme:ABC-DEF-ESX0

Additional Information

The NVMe namespaces and datastores can become invisible in the vSphere Client