Troubleshooting LUN Visibility Issues on ESXi Hosts After Migrating it from Another Storage Array.

Products

• VMware vSphere ESXi

Issue/Introduction

This article outlines troubleshooting steps for cases where LUNs are missing on ESXi hosts after being migrated from a different array. 

Symptoms: 

• After migrating from EMC Storage to HDS the LUNs are not detected on the host
• These LUNs are emulated from a different storage vendor (EMC in this case)
• The paths to these storage devices are marked as 'Dead'.
• When reviewing NMP errors in the vmkernel log, the following messages can be seen: 

2024-04-20T12:13:30.047Z cpu**:*******)WARNING: NMP: nmp_CallRegisterDevice:****: Device, seen through path vmhba*:C*:T*:L*** is not registered (no active paths)
2024-04-20T12:18:29.890Z cpu**:*******)WARNING: NMP: nmp_CallRegisterDevice:****: Device, seen through path vmhba*:C*:T*:L*** is not registered (no active paths)

Environment

VMware vSphere ESXi 7.0.x
VMware vSphere ESXi 8.0.x 

Cause

When Restarting / Power-On an ESXi host or rescanning a storage adapter, the host detects all physical paths to storage devices. Claim rules then dictate which multipathing module (such as NMP, HPP, or MPP) controls the paths for a specific device. The module that owns the device is responsible for managing multipathing support for it. 
In absence of a SATP claim rule where the storage device is not claimed by default claim rules available on the host

Resolution

• When setting up a new logical device or updating the hardware status of physical paths, NMP communicates with the Storage Array Type Plug-in (SATP). Additionally, the plug-in is notified when an I/O operation is about to be issued on a specific path
• When default settings for a specific device are inadequate, custom claim rules become necessary. By doing so, we can designate the storage path leading to the device as “unclaimed.” Afterward, a rescan of storage adapters ensures that the default claim rule is matched, and the Logical Unit Number (LUN) is once again managed by the Native Multipathing Plugin (NMP). In this scenario, manual addition of custom SATP rules is required each time a new host is added to the setup or environment
• Typically, SATP rules are essential for LUNs originating from different storage arrays. Even if they have the same emulation, additional custom SATP rules may be required. However, this applies specifically to underlying LUNs from different storage arrays.
• Compatibility between the ESXi host version and vendor generic SATP claim rules needs to be checked before for adding the custom SATP claim rule

Following are the necessary actions to address the issue: 

• Run this command to obtain the first common numbers of the naa.id of LUNs on a specific array:
  

  # esxcfg-mpath -l

• To check the existing path selection policy, run one of these commands:
  

  # esxcfg-info | grep -A1 "Default Path Selection Policy"
# esxcli storage nmp satp list

• To get the current SATP, run one of these commands:
  

  # esxcli storage nmp satp list
# esxcli storage nmp device list

For example: 

# esxcli storage nmp device list

naa.**********************:
 Device Display Name: ***** Fibre Channel Disk (naa.****************)
  Storage Array Type: VMW_SATP_DEFAULT_AA
 Storage Array Type Device Config: {action_OnRetryErrors=off}
 Path Selection Policy: VMW_PSP_RR
 Path Selection Policy Device Config: {policy=rr,iops=1,bytes=10485760,useANO=0; lastPathIndex=7: NumIOsPending=0,numBytesPending=0}
 Path Selection Policy Device Custom Config:
 Working Paths: vmhba**:C*:T*:L*, vmhba*:C*:T*:L*, vmhba**:C*:T*:L*, vmhba*:C*:T*:L*
   Is USB: false

• List the claim rule:
  

  # esxcli storage nmp satp rule list

For example: 
Name                 Device  Vendor    Model             Driver  Transport  Options                     Rule Group  Claim Options       Default PSP  PSP Options    Description
VMW_SATP_DEFAULT_AA   ***     Hu***    Inv***               **      ****                                    system      tpgs_off          VMW_PSP_RR   iops=1        ***

• To add the claim rule:
  

  # esxcli storage nmp satp rule add -s VMW_SATP_DEFAULT_AA -t vendor -V *** -M *** -c tpgs_off -P VMW_PSP_RR -O iops=1

-M|--model=string    Set the model string when adding SATP a claim rule.
-V|--vendor=string    Set the vendor string when adding SATP claim rules.

Note: It always recommended to migrate all the VMs from the host where changes being performed to other host with same version cluster.

• Confirm that the SATP rule is added successfully:

  # esxcli storage nmp satp rule list

For example:
# esxcli storage nmp satp rule list | grep EMC

• Get the Luns attached to host.
• Reboot the host.

 

Additional Information

Multiple configurations are supported by the family of arrays. Consequently, the ‘Claim Options’ value is set to ‘tpgs_off.’ If your device utilizes a generic SATP (such as ‘VMW_SATP_DEFAULT_AA’), one can create a claim rule specific to a storage array by matching the vendor’s name and/or model string. Additionally, if a Logical Unit Number (LUN) has been mapped to a host, you’ll need to restart the host. Keep in mind that new SATP rules take effect after modifying the Host Access Mode, but if you’re mapping the LUN for the first time, a restart isn’t necessary. 

Ref: Define NMP SATP Rules

External Ref: 
Changing an ESXi SATP Rule 
Removing an incorrect SATP Rule