vSAN Health Service - Performance Service - Stats master election check

Products

• VMware vSAN

Issue/Introduction

This article explains the Performance Service - ‘Stats master election’ check in the vSAN Health Service and provides details about why it might report an error.

Symptoms:
There should be only 1 host acting as the stats master keeping track of the performance of the entire vSAN cluster. The alert will show when there are no hosts acting as the stats master.

Environment

VMware vSAN 8.0.x
VMware vSAN 7.0.x
VMware vSAN 6.x

Resolution

Q: What does the Performance Service - Stats master election check do?

A: This check verifies that a Stats master exists in the cluster. The Stats master is responsible for collecting statistics from agent hosts.

In a healthy vSAN cluster with the performance service enabled only one Stats Master exists. The Stats Master must be a CMMDS master. In a partitioned vSAN cluster, multiple CMMDS masters can be present, but only one Stats master is allowed.

Q: What does it mean when it is in an error state and how do I troubleshoot the issue?

A: This check is in a Red error state if there is no Stats Master elected.

Check the health of the vSAN cluster by looking at vSAN Skyline Health ensuring all other health tests or green healthy state. Also, make sure that the performance service is enabled at vSAN> Configure> Services. 

If the cluster is healthy and not actively troubleshooting a performance issue you can try disabling/re-enabling vSAN Performance Service.

Note: Disabling/re-enabling Performance Service will wipe out any existing performance data.

 

In vCenter 8.0, to disable and re-enable vSAN Performance service, follow the below steps:

1. Log into RVC via vCenter SSH.
   1. Open vCenter SSH.
   2. Log into shell
   3. Run the command: rvc
   4. Login with administrator@<domain>@localhost
2. cd into localhost > datacenter > computers
3. Then run the command to delete the vSAN performance object: vsan.perf.stats_object_delete <cluster>
4. Once deleted, proceed to create the vSAN performance object by using the command: vsan.perf.stats_object_create <cluster>

 

You can also try restarting the vSAN management service on all hosts in the cluster by running:
/etc/init.d/vsanmgmtd restart

If all of these checks are confirmed healthy and the above troubleshooting steps doesn't clear the alert then collect the logs for the entire cluster including vCenter logs, file a Support Request and upload the logs for analysis.

Q: How to locate the elected stats master ESXi host in vCenter Web-Client?

A: Open the VCenter Web-Client by login as [email protected]. 

• Host and Cluster View >> Monitor >>  Skyline Health >> Performance Service >  Stats Primary Election 
• The right side pane will display the host which is currently working as stats Master

Additional Information

Impact/Risks:
Performance stats are not being collected for the cluster.