Datastore Connectivity Loss to IBM FlashSystem During Firmware Upgrade - ESXi

Products

• VMware vSphere ESXi

Issue/Introduction

This article addresses connectivity loss observed between ESXi hosts and IBM FlashSystem arrays during storage controller firmware upgrades. This behavior typically manifests during path discovery after a storage controller node reboot.

When upgrading storage firmware on an IBM FlashSystem (SAS-attached), the datastore may become inaccessible. The vmkernel.log may report the following warning:

WARNING: ScsiClaimrule: 1979: Path vmhba#:C0:T#:L0 is claimed by plugin NMP, but current claimrule number 65531 indicates that it should be claimed by plugin HPP.

Environment

VMware vSphere ESXi 8.0 and later Storage: IBM FlashSystem (SAS-attached)

Cause

This issue stems from two primary factors:

• Storage Firmware Defect: A known IBM storage firmware issue (Reference: SVAPAR-92804) may cause persistent loss of redundant paths where SAS direct-attached host paths are not recovered after a node reboot. "IBM Storage Virtualize Family Release Note 8.7.2.2" https://public.dhe.ibm.com/storage/san/sanvc/release_notes/872_releasenotes.html 
• Multipathing Claim Conflict: In vSphere 8.0 and later, the default claimrule logic selects the High Performance Plug-in (HPP) for SAS-connected storage. This conflicts with existing configurations where devices are expected to be managed by the Native Multipathing (NMP) plugin, causing path claiming errors during re-discovery.

Resolution

1. Update Storage Firmware

IBM has identified a defect (SVAPAR-92804) in specific firmware versions that prevents path recovery post-reboot.

1. Contact IBM Support to confirm if the current firmware version is affected.
2. Upgrade the storage firmware to version 8.7.2 or higher (or the appropriate backport) as recommended by IBM.
3. For defects and enhancements, subscribe to this article to receive updates on fix status (Reference: Subscribe to a Broadcom knowledge article).

2. Configure Custom Claimrule (Workaround)

If the firmware upgrade is not immediately possible, implement a persistent custom claimrule to force the use of the NMP plugin for IBM FlashSystem SAS devices.

1. Identify the vendor and model string: esxcli storage core device list -d <naa.ID>
2. Create a custom claimrule: esxcli storage core claimrule add -r 49 -t vendor -V <Vendor> -M <Model> -P NMP --force-reserved
3. Load the new claimrule: esxcli storage core claimrule load
4. Unclaim the affected path to trigger re-discovery using the new rule: esxcli storage core claiming unclaim -t path -p <runtime_name>

3. Performance Optimization

For IBM FlashSystem devices using the Round Robin (RR) path selection policy, apply the following settings for optimal performance:

1. Set IOPS=1 to ensure granular load balancing across all optimized paths: esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device <naa.ID>
2. Ensure useANO=0 (default) remains set to avoid routing I/O through non-optimized paths, which may increase latency.

Additional Information

For detailed steps on downloading and installing updates for Broadcom products, see Download Broadcom products and software

To speak with a customer representative or a Support Engineer see Contact Support. Scroll to the bottom of the page and click on your respective region.

For information on retrieving logs, see Collecting diagnostic information for VMware ESXi