vSAN Health Service shows vSAN Cluster Configuration Consistency - vSAN Performance Service in vSAN 8.0

Products

• VMware vSAN 8.x

Issue/Introduction

vSAN is showing an alarm for cluster configuration inconsistency for vSAN performance service in particular. Details show that one or more of the hosts are not configured for performance service. vSAN Health Service is showing inconsistency, however running health check shows cluster is good.

When looking at performance service it will show enabled, but say that the current datastore is not compatible with the storage policy assigned

---

 

Environment

VMware vSAN 8.x

Cause

While enabling vSAN performance service, it doesn't get enabled for every host in the cluster and becomes out of sync, though since 8.0 it is no longer possible to disable and then re-enable vSAN performance service via the vSphere GUI. 

This can also be caused if hosts in a cluster do not have NTP enabled and have large time skews across the cluster, as this confuses vsan health.

Resolution

If NTP is not set up and a time skew is present in the cluster, first correct that by enabling NTP on every host in the cluster.

Please see: ESX and ESXi host time keeping best practices

 

Then follow the rest of the steps:

vSAN performance service will need to be turned on via RVC. 

First log into RVC. Please see: Ruby vSphere Console Command Guide

SSH into vCenter Appliance as root

# rvc [email protected]@localhost 
(Note: local administrator account may be different from default, check if necessary and use that instead) 

It will then prompt for the local administrator password

Once into RVC navigate to the cluster you wish to enable vsan performance service on

Once in the cluster run the following command: 

# vsan.perf.stats_object_create . 
(Note: The . tells the command to run at the current directory, which in this case is the vSAN Cluster in question)

It will return *success* if completed. 

This will enable vSAN Performance Service on the hosts that were not configured yet and fix the cluster inconsistency. Run a new health check to verify issue is resolved. 

Still issue, then please recreate vSAN policy and run a health check again.

Additional Information

Note: If we have multiple vSAN Cluster in vcenter, we need to specify the cluster name against which we want to execute cmd "vsan.perf.stats_object_create"

For Example if we have 5 vSAN cluster under Datacenter and we want to execute the cmd against cluster name "Cluster_name_5" we need to run the cmd as below.

/localhost/Datacenter/computers> ls
0 Cluster_name_1 (cluster): cpu 1149 GHz, memory 7224 GB
1 Cluster_name_2 (cluster): cpu 0 GHz, memory 0 GB
2 Cluster_name_3 (cluster): cpu 241 GHz, memory 2037 GB
3 Cluster_name_4 (cluster): cpu 482 GHz, memory 4074 GB
4 Cluster_name_5 (cluster): cpu 1609 GHz, memory 10124 GB
/localhost/Datacenter/computers> vsan.perf.stats_object_info Cluster_name_5