Host Profile is non-compliant when using local SAS drives

Products

• VMware vSphere ESXi

Issue/Introduction

This problem occurs when you apply a host profile to a different host than the one from which you extracted the host profile. After you apply the host profile the PluggableStorageArchitectureProfile and/or the NativeMultiPathingProfile subprofiles of the storage host profile report that one or more storage devices are not compliant. You can not successfully remediate the host profile, no matter how many times you apply the profile.

Cause

Host Profiles enforce shared storage configurations across all hosts in the cluster and expect all devices to be either "local" or "shared clusterwide." The vSphere storage stack automatically detects which devices are local and makes this information available to higher level software but unfortunately this information is not always reliable for the special case of local Serial Attached SCSI (SAS) devices. Sometimes these devices are detected as non-local when they are in fact local.

If you have one or more SAS devices on the host from which you extracted the host profile that, although local to this host, are not detected to be local by the vSphere storage stack then this issue will occur. Alternatively you could have SAS devices that are shared among some but not all hosts in the cluster, which would also cause this issue. If you know the GUID for a local SAS, you can check if this is the case at the ESXi command line with the following command:

esxcli storage core device list --device device_guid | grep Local:

device_guid is either of the form naa.ddd where ddd are digits or mpx.vmhba0.C0:T0:L0 where the vmhba number is 1 or 2 digits and the C0:T0:L0 might be (for example) C1:T1:L1 or similar.

Environment

VMware vSphere ESXi 6.0

Resolution

You can perform one of the following four options to fix this issue.Each option requires that all hosts in your cluster have the same set of storage devices that are shared across the cluster, so ensure that the host from which you extract the host profile is properly configured as regards storage devices that are shared clusterwide. The local/non-local SAS issue will be fixed on ALL hosts regardless of which approach you use. If any shared storage devices are not present on other hosts you will see errors for those but not for any local SAS devices.

• If only some of your hosts have local SAS devices, then extracting the host profile from a host that you know has no local SAS device avoids this issue. Any (local) SAS devices found during profile application on other hosts are automatically configured to be not shared clusterwide which overrides any erroneous locality determination.

• Extract a host profile and then edit the extracted profile. For each local SAS device edit the "shared clusterwide" field and set the value to "False" (a SAS not properly detected as local sets this field to "True"). After editing the extracted profile, apply the profile to the host from which you extracted it in order to reset the "shared clusterwide" field on the host. This saves you time if you later need to re-extract the profile.

• Before extracting a host profile, change the device sharing at the command line on the vSphere host. To do this, run the following command for each local SAS device:
  esxcli storage core device setconfig --device device_guid --shared-clusterwide=false
  You can now extract the host profile.

• If you have a large number of local SAS devices on your hosts, run an automation script on one host. This step must be run on the vSphere host itself and requires that you pre-enable ssh access to a second host in the cluster and that the two hosts only share SAS devices that are shared by all hosts in the cluster. Log into the second host and run the following command:
  /bin/sharedStorageHostProfile automatic ip_address_of_second_host
  You must provide your login credentials to the second host and are asked to confirm the changes before committing them. Once completed, you can extract the host profile.
  

The profile from any of the above options should be applied to all other hosts in the cluster to reset the "shared clusterwide" field on the other hosts. Alternatively you can check compliance and then choose to remediate. Any shared storage devices which are not present on another host results in non-remediable compliance failures but the local SAS devices is reset such that compliance failure is remediated.