After upgrade to ESXi 8.0U3, "Lost access to volumes" on FC datastores (qedf)

Products

• VMware vSphere ESXi

Issue/Introduction

If you are using the qedf (Qlogic) drivers, after upgrading to ESXi 8.0 U3, you may lose access to volumes or your disks may appear degraded. This can impact the applications on the virtual machines. 

In var/log/vmkernel.logs, these similar errors are observed in the logs. 

Unable to map SG array on path vmhba##:C#:T#:L#. Status: DMA mapping could not be completed DMA Error: Can't meet SG element alignment.

You may also experience the following errors around the same time as as unable to map SG array on path messages:

Lost access to volume ########-########-####-##########98 (datastore) due to connectivity issues. Recovery attempt is in progress and outcome will be reported shortly.

or

qedf:vmhba#:qedfc_eh_abort:####:Info: IO not found

Environment

VMware vSphere ESXi 8.0 update 3

Cause

DMA allocation optimization controlled by the ESXi "ScsiPathSplitUseSimpleCloneBuffer" parameter, violates the DMA constraints of the qedf driver. This causes aborts from the drivers leading to disruptions in storage connectivity 

Resolution

This issue is scheduled to be fixed in 9.0 U1 release.

However, to work around this, set ScsiPathSplitUseSimpleCloneBuffer parameter to 0: 

 

1. List the ScsiPathSplitUseSimpleCloneBuffer configuration: 
   esxcli system settings advanced list -o /Scsi/ScsiPathSplitUseSimpleCloneBuffer
      Path: /Scsi/ScsiPathSplitUseSimpleCloneBuffer
      Type: integer
      Int Value: 1 <<<<==== The default value of this config option
      Default Int Value: 1
      Min Value: 0
      Max Value: 1
      String Value:
      Default String Value:
      Valid Characters:
      Description: Use simple clone buffers to split IOs at path layer whenever possible.
      Host Specific: false
      Impact: none
   
   
2.  Set the parameter to 0:
   esxcli system settings advanced set -o /Scsi/ScsiPathSplitUseSimpleCloneBuffer -i 0
   
3. Verify the change: 
   esxcli system settings advanced list -o /Scsi/ScsiPathSplitUseSimpleCloneBuffer
      Path: /Scsi/ScsiPathSplitUseSimpleCloneBuffer
      Type: integer
      Int Value: 0 <<<<==== This value is reset to 0 after step #2
      Default Int Value: 1
      Min Value: 0
      Max Value: 1
      String Value:
      Default String Value:
      Valid Characters:
      Description: Use simple clone buffers to split IOs at path layer whenever possible.
      Host Specific: false
      Impact: none

 

No reboot is required and changes will be persistent through a reboot.