The NGUID-only NVMe device is detected as a snapshot after upgrading ESXi from 6.5 U3.

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

After upgrading from ESXi 6.5U3 to subsequent ESXi releases, the NGUID-only NVMe device will be detected as a snapshot and the VMFS datastore created on this device cannot be seen.

In kernel log, there are messages similar to:
2023-06-30T06:20:32.146Z cpu6:2097511)LVM: 11279: Device eui.cdaa3cc33c65d4b1000c296d378b6252:1 detected to be a snapshot:
2023-06-30T06:20:32.146Z cpu6:2097511)LVM: 11286:   queried disk ID: <type 1, len 44, lun 0, devType 0, scsi 0, h(id) 17764953765328980470>
2023-06-30T06:20:32.147Z cpu6:2097511)LVM: 11293:   on-disk disk ID: <type 1, len 44, lun 0, devType 0, scsi 0, h(id) 8147516696569718987>

Environment

VMware vSphere ESXi 6.5

Cause

ESXi storage stack started to support NGUID since ESXi 6.7U1. For NGUID-only NVMe devices, the device name should be t10.xxx in ESXi 6.5U3 and eui.xxx in ESXi 6.7U1 and later. Since NVMe driver's NGUID support is merged to ESXi 6.5U3 by mistake, the device name in ESXi 6.5U3 is shown as eui.xxx. As a result, when upgrading 6.5U3 to subsequent ESXi releases, the NGUID-only NVMe device will be detected as a snapshot and the VMFS datastore created on this device cannot be seen.

Use the following command to check whether an NVMe device is NGUID-only:
# esxcli nvme device namespace get -A <vmhba_name> -n <namespace_id>
Namespace Globally Unique Identifier: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx ----------------------> Non-zero value
IEEE Extended Unique Identifier: 0000000000000000 -------------------------------------------> All-zero value

Resolution

Currently there is no resolution to the issue.

Workaround:
To workaround the issue, please refer to Troubleshooting LUNs detected as snapshot LUNs in vSphere for handling the snapshot volume.