Replica usage is higher than expected on vSAN OSA Stretched Cluster.

Products

• VMware vSAN
• VMware vSAN 8.x

Issue/Introduction

On a vSAN Stretched Cluster, you may notice that the "Replica" consumption in your usage breakdown is much higher than the "Primary Data".

Environment

VMware vSAN OSA 8.x 

Cause

 

How vSAN distinguishes between data copies:

When monitoring a vSAN Stretched Cluster, you may observe that Replica Usage in the capacity breakdown is significantly higher than Primary Data (the actual data written by the VM).

This is expected behavior due to how vSAN calculates data copies in a stretched cluster environment.The Role of Primary Data vs. Replica Usage

vSAN distinguishes between data copies as follows:

• Primary Data: The first copy of a data block is always labeled as Primary Data.
• Replica Usage: Every additional copy of a data block required by your storage policy (for both site-level and local-level protection) is labeled as Replica Usage.

 

How Data Protection is applied in a Stretched Cluster Policy:

A vSAN Stretched Cluster Storage Policy is designed to protect your data at two different levels :

1. 1. PFTT (Primary Failures To Tolerate/Site Disaster tolerance): Protects against an entire site going down.

   2. SFTT (Secondary Failures To Tolerate): Protects against a local disk or host failure within a single site.

The high replica consumption is caused by the multiplication of data copies needed to satisfy both the PFTT and SFTT requirements.

 

 

The "Replica" Effect on Capacity:

If you have a policy set to PFTT=1 (Stretched) and SFTT=1 (RAID-1 Mirroring):

• • vSAN creates 2 copies to satisfy the Stretched (PFTT) requirement (one for Site A, one for Site B).

  • vSAN then mirrors each of those copies locally to satisfy the SFTT requirement.

  • Result: You have 4 total copies of every data block.

This is why your replica usage is significantly higher; for every 1 TB of actual data you write, the system may be storing 3 TB of replicas to ensure it can survive both a site failure and a local hardware failure.

Resolution

vSAN is performing exactly as configured to ensure high availability.

This higher replica usage the therefore expect and as per design this "high replica usage" is a result of the defined storage policies.

vSAN Stretched Cluster Guide

 

However, you can manage this consumption by reviewing your policies:

• Check if all VMs require "Stretched" protection.
  If not an option could be to use "Pinned" policies (site-local) consume significantly less replica space because they don't mirror data to the secondary site.
• If you have further capacity concerns Resolving Storage Capacity Issues in a vSAN Datastore / VMs are not running despite free space on vSAN Datastore