Proactive Rebalance: Running a manual rebalance may be necessary when your vSAN cluster is imbalanced. This operation moves components from the over-utilized disks to the under-utilized disks. When performing a manual rebalance, this operation runs for 24 hours and then stops.
Note: Running a manual rebalance utilizes some system resources and this process can take several hours to complete. This depends on the number objects that needs to be rebalanced to reduce disk usage variance across cluster.
It is recommended to run Proactive rebalance when there is minimal workload by monitoring vSAN performance charts.
To run a Proactive Rebalance in vSphere 6.7 U2 and lower:
- Navigate to the vSAN cluster in the vSphere Web Client.
- Click the Monitor tab and click vSAN.
- Click Health.
- In the vSAN health service table, select Warning: Virtual SAN Disk Balance. You can review the disk balance of the hosts.
- Click the Rebalance Disks button to rebalance your cluster.
Note: This task may take many hours.
--------------------------------------------------------------------------------------------------------------------
Automatic Rebalance:
Starting in vSAN 6.7 U3, disk rebalancing is no longer a manual method, and it needs to be enabled as a service within vSAN cluster settings (explained below). If this is not enabled, vSAN will only initiate rebalance on vSAN disks when any of the vSAN disks crosses 80% Capacity threshold.
Note: Disk rebalancing can impact the I/O performance of your vSAN cluster. To avoid this performance impact, you can alter threshold value or turn off automatic rebalance when peak performance is required.
Procedure to configure Automatic Rebalance:
1. Navigate to the vSAN cluster.
2. Click the Configure tab.
3. Under vSAN, select Services.
4. Click to edit Advanced Options.
5. Click to enable or disable Automatic Rebalance.
6. Set the variance threshold to any percentage from 20 to 75 as per your requirement.
The threshold to initiate rebalance is set to 30% by default, which means that if any two disks have this variance (one is 30% more loaded than the other), rebalancing of components begins. Rebalancing will continue until the variance reaches half of the set threshold value, i.e., 15% by default (or until Automatic Rebalance is disabled).
There is also a health check for vSAN Disk Balance, where you can see disk usage details of the vSAN cluster. If Automatic Rebalance is enabled, vSAN automatically tries to keep this health check green. If it is disabled, this health check is triggered and will requires the admin to manually trigger a Rebalance Disks task, or re-enable Automatic Rebalance.
The toggle for enabling or disabling this cluster-level feature can be found in vCenter, under
Configure > vSAN > Services > Advanced options > “Automatic Rebalance” as shown in Figure 1.
RECOMMENDATION: Keep the “Rebalancing Threshold %” entry to the default value of 30. Decreasing this value could increase the amount of resynchronization traffic and cause unnecessary rebalancing for no functional benefit.The “vSAN Disk Balance” health check was also changed to accommodate this new capability. If vSAN detects an imbalance that meets or exceeds a threshold while automatic rebalance is disabled, it will provide the ability to enable the automatic rebalancing, as shown in Figure 2. The less-sophisticated manual rebalance operation is no longer available.
Once the Automatic Rebalance feature is enabled, the health check alarm for this balancing will no longer trigger and rebalance activity will occur automatically.
Should Automatic Rebalancing Be Enabled?
Yes, it is recommended to enable the automatic rebalancing feature on your vSAN clusters. When the feature was added in 6.7 U3, VMware wanted to introduce the capability slowly to customer environments and remains this way in vSAN 7. With the optimizations made to our scheduler and resynchronizations in recent editions, the feature will likely end up enabled by default at some point.
There may be a few rare cases in which one might want to temporarily disable automatic rebalancing on the cluster. Adding a large number of additional hosts to an existing cluster in a short amount of time might be one of those possibilities, as well as perhaps nested lab environments that are used for basic testing. In most cases, automatic rebalancing should be enabled.
Links
Document:
https://docs.vmware.com/en/VMware-vSphere/6.5/com.vmware.vsphere.virtualsan.doc/GUID-968C05CA-FE2C-45F7-A011-51F5B53BCBF9.htmlDocument:
https://docs.vmware.com/en/VMware-vSphere/6.7/com.vmware.vsphere.vsan-monitoring.doc/GUID-968C05CA-FE2C-45F7-A011-51F5B53BCBF9.htmlRelease Notes:
https://docs.vmware.com/en/VMware-vSphere/6.7/rn/vmware-vsan-67u3-release-notes.html
Workaround:
Unfortunately, if the automatic rebalance is not working, this means that vSAN is not finding the way to rebalance the objects without fulling other disks. Take into consideration that is not only moving data, vSAN needs to ensure the accessibility of the objects.
Some options to validate are the following:
- Clean up space, look for VMs that are not in use or big files that can be deleted.
- Add more capacity disks to the hosts.
- Look up for VMs that are not critical or used for test and change the policy to RAID 0.
- Check if some VMs with default policy can be changed to RAID 5 or RAID 6 erasure coding.
- You can enable dedup and compression on the cluster to eliminate duplicate data.