"Unable to map SG array on path vmhba" and "lost volume access" on ESXi 8.0 U3 with qedf driver

Products

• VMware vSphere ESXi

Issue/Introduction

If you are using the qedf (Qlogic) drivers, after upgrading to ESXi 8.0 U3, you may lose access to volumes or your disks may appear degraded. This can impact the applications on the virtual machines. 

In var/log/vmkernel.logs, these similar errors are observed in the logs. 

Unable to map SG array on path vmhba##:C#:T#:L#. Status: DMA mapping could not be completed DMA Error: Can't meet SG element alignment.

You may also experience the following errors around the same time as as unable to map SG array on path messages:

Lost access to volume ########-########-####-##########98 (datastore) due to connectivity issues. Recovery attempt is in progress and outcome will be reported shortly.

or

qedf:vmhba#:qedfc_eh_abort:####:Info: IO not found

Environment

• VMware ESXi 8.0 Update 3

• QLogic FastLinQ QL41xxx Series 10/25 GbE Controller (FCoE)

• qedf driver

Cause

This issue is caused by a conflict between the DMA constraints exposed by the qedf driver and the Pluggable Storage Architecture (PSA) logic. When I/O requests with large block sizes are issued, they may be split incorrectly based on these constraints, leading to DMA mapping failures and subsequent loss of volume access.

On ESXi 8.0.3 , DMA allocation optimization for large blocks controlled by the ESXi adv configure "ScsiPathSplitUseSimpleCloneBuffer" parameter.

Resolution

ESXi code enhancement to address the issue has been planned for 9.0 U1 release.

To workaround this issue, modify the SCSI path split configuration on the affected ESXi hosts.

1. Verify the current value of the ScsiPathSplitUseSimpleCloneBuffer setting (Default is 1): 
   esxcli system settings advanced list -o /Scsi/ScsiPathSplitUseSimpleCloneBuffer
      Path: /Scsi/ScsiPathSplitUseSimpleCloneBuffer
      Type: integer
      Int Value: 1 <<<<==== The default value of this config option
      Default Int Value: 1
      Min Value: 0
      Max Value: 1
      String Value:
      Default String Value:
      Valid Characters:
      Description: Use simple clone buffers to split IOs at path layer whenever possible.
      Host Specific: false
      Impact: none
   
   
2.  Disable the setting by changing the value to 0:
   esxcli system settings advanced set -o /Scsi/ScsiPathSplitUseSimpleCloneBuffer -i 0
   
   
3. Verify the change was successful (Ensure the Int Value is 0) :
   esxcli system settings advanced list -o /Scsi/ScsiPathSplitUseSimpleCloneBuffer
      Path: /Scsi/ScsiPathSplitUseSimpleCloneBuffer
      Type: integer
      Int Value: 0 <<<<==== This value is reset to 0 after step #2
      Default Int Value: 1
      Min Value: 0
      Max Value: 1
      String Value:
      Default String Value:
      Valid Characters:
      Description: Use simple clone buffers to split IOs at path layer whenever possible.
      Host Specific: false
      Impact: none

 

No reboot is required and changes will be persistent through a reboot. 

Additional Information

• This issue has not been observed on hosts using Emulex adapters (lpfc driver) or QLogic adapters utilizing the qlnativefc driver.

• Subscribe to this knowledge article to get updates on this issue.