Warning: "Virtual disk 'Hard disk' is a mapped direct-access LUN that is not accessible." while migrating or powering on VM with RDMs attached.
search cancel

Warning: "Virtual disk 'Hard disk' is a mapped direct-access LUN that is not accessible." while migrating or powering on VM with RDMs attached.

book

Article ID: 318949

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

Symptoms:

  • Powering on or migrating (vMotion) a virtual machine with a raw device mapping (RDM) may report compatibility issues as below.

Virtual disk 'Hard disk xx' is a mapped direct-access LUN that is not accessible. 
In order to proceed with the operation, the target disk's datastore and format must be different than that of the source disk.
pxx.####.com

  • During the migration task, when you select the destination host to migrate the VM, the below warnings show up in the migration task wizard.

Environment

VMware vSphere ESXi 6.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

This is an informational event which indicates that the host attempting to start or receive the virtual machine cannot access the device (LUN) backing the raw disk mapping. For more information on raw device mappings, see About Raw Device Mapping.

The cause for the warnings is as below.

The RDM LUNs were presented with HLUN or SCSI ID '78' today and attached to the VM which created a RDM pointer file using the current VML ID.  The VM is powered on and running. The HLUN / SCSI ID from the storage changed for the RDM LUN after a few days to a different number 'L114'. This would change the VML ID to match with a new LUN Number upon rescan of storage devices on ESXi hosts. However, already created RDM pointers will still see the old VML ID for the running virtual machine.

You will see VM with different VML ID  and ESXi hosts storage device section shows different VML ID. 

For RDM LUN: naa.600507680c848053a0000000000025ae

vml.0200720000600507680c848053a0000000000025ae323134352020 - Hosts VML shows currently
vml.02004e0100600507680c848053a0000000000025ae323134352020 - VML ID shows on the host where VM is running or VM settings under RDM drive.

The below LUN decoding from VML shows two different HLUN/SCSI ID for the same RDM LUN.

vml.0200 72 0000 600507680c848053a0000000000025ae323134352020 - HEX 72 = 114 (HLUN/SCSI ID from storage)
vml.0200 4e 0100 600507680c848053a0000000000025ae323134352020 - HEX 4e = 78 (HLUN/SCSI ID from storage)

If the logs available for previous/older dates, you may check for the LUN NAA ID on /var/run/log/vmkernel.log or /var/run/log/vmkwarning.log and you can observe the different information for the same device as shows in the below example.

2024-10-29T10:49:14.792Z: [scsiCorrelator] 15300917400768us: [esx.problem.storage.redundancy.degraded] Path redundancy to storage device naa.600507680c848053a0000000000025ae degraded. Path vmhba5:C0:T13:L78 is down. Affected datastores: Unknown.

2024-11-05T20:49:44.247Z: [scsiCorrelator] 15941746856030us: [esx.problem.storage.redundancy.degraded] Path redundancy to storage device naa.600507680c848053a0000000000025ae degraded. Path vmhba3:C0:T23:L114 is down. Affected datastores: Unknown.

 

A raw device mapping file contains metadata, specifically SCSI vital product data (VPD) page 0x83, for uniquely identifying a device. A host attempting to access the physical LUN reads the mapping file metadata, and searches the devices known to the host for a LUN with the same VPD page 0x83 information.

The contents of SCSI VPD page 0x83 should match when requested by all hosts in a vSphere cluster, and the RDM should be accessible by all hosts in the cluster. However, discrepancies in device presentation between hosts in a cluster may lead to different hosts receiving different contents for VPD page 0x83 for the same device. In this case, hosts which did not create the RDM mapping file may be unable to locate the raw device mapping LUN when powering on or migrating a virtual machine.

Notice: Cold migrations of powered off virtual machines do not require access to the raw device mapping. Symptoms may be noticed after cold migration, when the virtual machine is powered on.

Notice: In the case of vMotion for a virtual machine with an RDM device, even if the same device and LUN are presented on both the source and destination hosts, the vMotion compatibility check may fail if the VML ID is different.  This issue is caused by an omission in the compatibility check logic. It has been fixed in vSphere 8.0U3 (vCenter Server 8.0 U3/ESXi 8.0 U3).

Resolution

Identifying mis-presented devices using VML identifiers

The VPD page 0x83 information, when available, forms the basis of the unique VML identifier which the ESX/ESXi host assigns to each storage device. This VML identifier should be consistent for a given device across each host in a vSphere cluster.

Identify the device being used for a raw device mapping by its VML identifier, such as by reviewing the contents of /vmfs/devices/disks/, on each host in a cluster. Compare the VML identifier obtained for the device on each host. For more information, see Identifying disks when working with VMware ESX (1014953).

The highlighted sections of the VML identifiers in this example do not match between the hosts, indicating a LUN with a different presentation to each host:

  • Host 1:

    vml.020005010060060480000190104063533030353###53594d4d4554 -> naa.60060480000190104063533030353###
     
  • Host 2:

    vml.02004d000060060480000190104063533030353###53594d4d4554 -> naa.60060480000190104063533030353###

Note: In the above example, the highlighted value equates to the device's LUN number represented in hexadecimal.

On host 1, the LUN has been presented as LUN 261(0x105), and on Host 2, it has been presented as LUN 77(0x4d).

LUN presentation (including LUN numbering) should be made consistent for every host participating in a cluster that could run the virtual machine. The raw device mapping metadata file should be consistent with that presentation, and vCenter Server's cache of this information should be accurate.

Segmented cluster:

If one segment of a vSphere Cluster has correct presentation of the LUN being used for a raw device mapping, but other segments do not, the virtual machine operates correctly only within the segment with correct presentation. Correct the presentation in the nonworking segment(s):

  1. Unpresent the LUNs from the hosts with incorrect presentation. For more information, see How to detach a LUN device from ESXi hosts.
  2. Represent the LUNs in the same manner as the working hosts. For more information, engage your storage array vendor.
  3. Rescan to pick up storage changes. For more information, see Performing a rescan of the storage (308546).
  4. Verify that the host LUN ID matches on all the hosts in the cluster that have access to that LUN.
  5. Verify that the VML ID for the device matches on all the hosts in the cluster.
  6. If the host LUN ID matches and the VML ID does not match:
     
    1. Unpresent the LUN on the host where the VML ID does not reflect the correct host LUN ID. For more information, see How to detach a LUN device from ESXi hosts.
    2. Perform a rescan, then represent the LUN. Verify that the VML ID matches.
       
  7. Attempt migration of the virtual machine from the working segment to the corrected hosts.
Note: The above procedure may result in the VML mismatch issue still being present due to the way ESX caches vml devices. If the issue persists, repeat the procedure, but this time represent the LUN from the array with a different LUN ID. This forces a new VML identifier to be created for the LUN.

Note: An alternate method of correcting VML IDs for mispresented LUNs:
  1. Note the NAA_ID of the LUN.
  2. Detach RDM using vSphere client.
  3. Un-present the LUN from host on array.
  4. Rescan host storage.
  5. Remove LUN from detached list using these commands:

    esxcli storage core device detached list
    esxcli storage core device detached remove -d NAA_ID
     
  6. Rescan the host storage.
  7. Re-present LUN to host.
  8. Rescan host
If the LUN has been flagged as perennially reserved, this can prevent the removal from succeeding.

Run this command to remove the flag:
# esxcli storage core device setconfig -d NAA_ID --perennially-reserved=false

Now the command to remove the device should work.
# esxcli storage core device detached remove -d NAA_ID

There is no harm in setting the flag to false, even if it is not needed, so it might be simpler to always run both commands.

If the RDM LUN has been flagged as perennially reserved base below guidelines:
Microsoft Windows Server Failover Clustering (WSFC) with shared disks on VMware vSphere 7.x: Guidelines for supported configurations
Remember to set perennially-reserved flag to true after Re-present LUN to host.
Perennial reservations on the clustered VMDK enabled datastores are required for all ESXi hosts hosting VM nodes with pRDMs and Clustered VMDKs.

# esxcli storage core device setconfig -d NAA_ID --perennially-reserved=true

Whole cluster:

If a virtual machine cannot be started on any host in a vSphere Cluster, the presentation to each host may differ from the VPD Page 0x83 information stored within the raw device mapping file. Recreate the mapping file:

  1. Power down the virtual machine.
  2. Right-click the virtual machine and click Edit Settings.
  3. Make note of the raw device mapped disks which are attached to the virtual machine.
  4. Remove the raw device mapped disks from the virtual machine.
  5. Click OK.
  6. Right-click the virtual machine and click Edit Settings.
  7. Click Add to re-add the previously removed raw device(s).
  8. Click OK.
  9. Power on the virtual machine and re-attempt the operation which previously failed.

vCenter Server cache:

The vCenter Server maintains a cache of storage topology information for virtual machines. If this information has become out of sync with the presentation, power operations or migrations initiated from vCenter Server may fail. Remove the affected virtual machine from inventory and re-register it to clear the cached information:

  1. Power down the virtual machine.
  2. Right-click the virtual machine and remove it from the inventory.
  3. Re-register the virtual machine to the inventory. For more information, see How to register or add a Virtual Machine (VM) to the vSphere Inventory in vCenter Server.
  4. Power on the virtual machine and retry the operation which previously failed.

Additional Information

Powered by Wolken Software