Determining the current vSAN Cluster's CMMDS and Stats Master/Primary host

Products

• VMware vSAN

Issue/Introduction

Every vSAN Cluster has a CMMDS/Stats Master/Primary and Backup ESXi host that manages the cluster. This master/primary host is responsible for the following:

• Recovery and maintenance of a clusters networked node members.
• It manages the inventory of items such as host nodes, devices, and networks.
• It also stores metadata information, such as policies and RAID configuration for vSAN objects.
• It's also responsible for collecting the clusters performance metrics to monitor the clusters overall performance.

Below are examples when VMware by Broadcom Support may ask for just the master/primary host logs and/or a subset of cluster logs to be collected, especially for large clusters:

• When investigating vSAN Cluster performance issues it's crucial to get the master node logs collected first to ensure the performance issue is captured at the time of the event.
• Troubleshooting Stats Master Election failures - at minimum just the CMMDS master node may be enough but it's always best to collect the entire cluster logs
• Hosts in the cluster not contributing performance stats - at minimum just the master node and the hosts not contributing perf stats may be enough

 

Environment

VMware vSAN (All Versions)

Cause

 

 

 

Resolution

• From vCenter 7.x vSAN Skyline Health test "Stats master election" displays the CMMDS/Stats master host name.
  
  

• For vCenter 8.X vSAN Cluster -> Host and Cluster view  -> Monitor -> vSAN -> Skyline Health -> Retest ->  All -> Use filter  "Stats Primary Election"
   
  
• If vCenter is down or vSAN Health is not loading you can run the below script from any host in the cluster to get the master/backup host info
  
  

  echo -e "\n--- vSAN Cluster Roles ---"; MASTER=$(esxcli vsan cluster get | awk -F ': ' '/Sub-Cluster Master UUID/ {print $2}'); BACKUP=$(esxcli vsan cluster get | awk -F ': ' '/Sub-Cluster Backup UUID/ {print $2}'); HOST_MAP=$(cmmds-tool find -f json -t HOSTNAME | grep -E "uuid|content" | sed 'N;s/\n/ /' | awk -F \" '{print $10": " $4}'); echo -e "Master Node:\t$(echo "$HOST_MAP" | grep "$MASTER")"; echo -e "Backup Node:\t$(echo "$HOST_MAP" | grep "$BACKUP")"

  --- vSAN Cluster Roles ---
Master Node:    esxi03: 69c38079-d91a-acdc-b1as-############
Backup Node:    esxi13: 69c38078-7404-7c5c-23cd-############

Additional Information

vSAN Health Service - Performance Service - Stats primary election check
Collecting vSAN Performance Service data for vSAN performance issues
vSAN Health Service - Performance Service - All hosts contributing stats