Proactive Rebalance does not balance disk usage in stretched clusters using Storage Policies with only local protection (PFTT=0,SFTT=1).
search cancel

Proactive Rebalance does not balance disk usage in stretched clusters using Storage Policies with only local protection (PFTT=0,SFTT=1).

book

Article ID: 326802

calendar_today

Updated On:

Products

VMware vSAN

Issue/Introduction

This article provides the steps used to identify the cause of this issue and remediate this condition.

Symptoms:
Proactive Rebalance does not balance disk usage in stretched clusters using Storage Policies with only local protection (PFTT=0,SFTT=1).

Environment

VMware vSAN 6.7.x
VMware vSAN 6.6.x

Cause

When vSAN data-Objects have a Storage Policy applied which only has local-protection rules (e.g. PFTT=0,SFTT=1,Affinity=Preferred) these Objects data-components can only reside on one data-site in the cluster and this can result in imbalance in disk usage on each site. 

For example: if 20TB of data has Affinity to Preferred site and only 10TB of data has Affinity to Secondary site then there will be a disparity of 10TB used space between the sites which Proactive Rebalance cannot remediate (as moving components to the lower utilised site would violate the rules of the Storage Policy).

Resolution

Identify that the disparity in disk usage is between the sites, this can be achieved via RVC by checking the disk usage of the nodes on each site using:

> vsan.disks_stats <pathToCluster>

If one site has all the disks exceeding the max variance in disk-usage (30%), it should also show only data to be moved from the site with the higher usage when checked via RVC:

> vsan.proactive_rebalance_info <pathToCluster>

Further clarification of which VMs have which Storage Policies applied can be performed in bulk via RVC:

> spbm.check_compliance <pathToVMs>*

*Note that VMs can reside in many locations if Resource Pools are used and thus using /*/* against multiple pools may be required in order to identify all VMs in inventory.

From the vSphere Clients it is possible to retrieve information relating to how much space each VM using a Storage Policy is consuming:
Home > Policies and Profiles > VM Storage Policies > Policy Name > VMs > Virtual Machines (sort by 'Used Space')

To balance the data usage evenly across the sites it is advised to identify VMs that do not have a business requirement to reside on any specific site and apply a Storage Policy with affinity to the lower utilised site. This will move the imbalanced data to the other site thus balancing the clusters disk-usage. If there is a substantial quantity of data to be moved, consider performing this task during off-peak hours so as to avoid resource contention.

How to get access into RVC. 

1. SSH into vCenter and type either rvc localhost or rvc, depending on how the user account has been set,  then enter proper user name and its corresponding password 
2. Navigate through the menu using ls and cd    
3. From the respective path you can execute the above commands as shown on below screenshots. 
4. On this case the commands are being executed against SDDC-Cluster1 cluster  on SDDC-Datacenter data center












See also
vSAN Stretched Cluster Site Imbalance (78045)
https://ikb.vmware.com/s/article/78045