Storage Policy for vSAN Stretched Cluster shows incompatible

Products

• VMware vSAN

Issue/Introduction

A storage policy with Secondary failures to tolerate shows as incompatible for a stretched cluster's datastore though it meets all requirements. 

The error showing in vCenter when looking under Storage Compatibly says 'Datastore does not match current VM policy. "Secondary level of failures to tolerate is not supported by this cluster" '


This can also be seen  when trying to mount the datastore from the stretch cluster to another cluster as a vSAN remote datastore with the following check failing "The remote vSAN datastore can provision objects with its default policy'"





In the /var/log/clomd.log on the ESXi hosts you will see the following messages:

Er(27) clomd[8935223]: [Originator@6876] CLOM_GetNumSlackSpaceHostsInPreferredSite: Can't find preferred FD: Failure.
Er(27) clomd[8935223]: [Originator@6876] CLOM_GetNumSlackSpaceHostsInNonPreferredSite: Can't find non-preferred FD: Failure.
Er(27) clomd[8935223]: [Originator@6876] CLOM_GetNumSlackSpaceHostsInPreferredSite: Can't find preferred FD: Failure.
Er(27) clomd[8935223]: [Originator@6876] CLOM_GetNumSlackSpaceHostsInNonPreferredSite: Can't find non-preferred FD: Failure.
Er(27) clomd[8935223]: [Originator@6876] CLOM_GetNumSlackSpaceHostsInPreferredSite: Can't find preferred FD: Failure.
Er(27) clomd[8935223]: [Originator@6876] CLOM_GetNumSlackSpaceHostsInNonPreferredSite: Can't find non-preferred FD: Failure.

Environment

vSAN 8.0
vSAN 9.0
vSAN HCI Mesh
vSAN stretched cluster 

Cause

This is caused by the setting "stretchedClusterMode" being set to 0 on the hosts of the stretch cluster that is hosting the datastore. 
This setting should be a 1 for stretched cluster and the datastore hosted on the stretch cluster to be able to use a storage policies with  a secondary level of failures to tolerate
You can check this setting with the command

vsish get /vmkModules/vsanutil/stretchedClusterMode

If this returns a value of 0 it confirms the setting is incorrect 

Resolution

Set "stretchedClusterMode" setting to 1 using the following command on all hosts

vsish set /vmkModules/vsanutil/stretchedClusterMode 1

Confirm the setting with the command 

vsish get /vmkModules/vsanutil/stretchedClusterMode

This should now return a value of 1 on all hosts 

Once all hosts have the value set to 1 it should resolve the behavior. In some cases vCenter may need the vsan health service restarted using the following commands

vmon-cli -k vsan-health
vmon-cli -r vsan-health

These actions has no impact on production and running VMs
Restarting the vSAN management services though vCenter may disrupt vCenter management operations for a short time.