The consumed storage of replication disk stored in vSAN cluster is significantly larger compared to source VM disk capacity

Products

• VMware Cloud Director

Issue/Introduction

• On-Prem to Cloud replication increased vmdk disk storage consumption significantly.
• The destination site uses VSAN storage.
• After the initial sync the VM had 50 GB of consumed storage. Then after the first replica completed 24 hours later, the consumed storage has increased to over 100 GB.

Environment

VMware Cloud Director Availability 4.7.3.x

Cause

This issue occurs when vSAN is used as the storage type on the destination. vSAN stores the vmdk file on the VSAN datastore as an Object.  The relationship between Objects, Components, and Failures to Tolerate (FTT) defines how data is distributed across the vSAN cluster to ensure availability. The amount of components depends on the configured FTT (RAID) on the VSAN storage policy.  Additionally factors such as vSAN object space reservation settings and vSAN architecture (ESA/OSA) choice will also influence.

VMware Cloud Director Availability(VCDA) itself does not have any independent logic to calculate the consumed storage. It simply reports the consumed storage size reported to it by vCenter Server. The size accurately reflects the total vSAN datastore footprint for the VMDK, including fault tolerance overhead, rather than the raw guest OS disk size.

vSAN reports the used storage of the Object to vCenter Server. vCenter Server then reports this value to VCDA and this is what is reflected as the consumed storage value. VCDA then informs Cloud Director about the size and this is reflected as the storageConsumedMb value on the storage policy. 

Resolution

This is expected behavior in VCDA when using VSAN storage on the destination site. 

To reduce the reported consumed storage then modification would be required at the vSAN level. For example the vSAN storage policy could be modified to utilize a lower FTT or a more space-efficient RAID configuration, provided the cluster architecture supports it.

To confirm the vSAN configuration and reporting:

1. Login to vCenter Server.
2. Click the menu list in the top left and navigate to 'Policies and Profiles'.
3. On the 'VM Storage Policies' page select the vSAN storage policy in use.
4. Verify the configured Failures to Tolerate (FTT) / RAID settings within the assigned vSAN storage policy.
5. Next open an SSH session to an ESXi host within the vSAN cluster.
6. Inspect the C4 vmdk file to obtain the object uuid using the cat command. 
   
   Example: cat /vmfs/volumes/vsan-datastore-name/vm-name/vm-name.vmdk

7. In the output of the vmdk file, the 'Extent description' contains the UUID. It will be third number on that line.

   Example: The UUID in this example is: 9876####-####-####-####-########

   RW 2097152 VMFS "vsan://123#####-####-####-########/9876####-####-####-####-########"

8. Using the object ID, inspect the object size:
   
   esxcli vsan debug object list -u 9876####-####-####-####-########"
   
   
9. In the output, the reported used size is confirmed.
   
   Object UUID: 9876####-####-####-####-########
   Version: 10
   Health: healthy
   Owner: <hostname>
   Size: 50.00 GB
   Used: 101 GB
   Used 4K Blocks: 99 GB
   Policy:
      stripeWidth: 1
      cacheReservation: 0
      proportionalCapacity: 0
      hostFailuresToTolerate: 1
      forceProvisioning: 0
      spbmProfileId: #########
      spbmProfileGenerationNumber: 2
      storageType: Allflash
      replicaPreference: Capacity
      iopsLimit: 0
      checksumDisabled: 0
      subFailuresToTolerate: 1
      CSN: 40
      spbmProfileName: #########
      locality: None

Additional Information

For more information regarding vSAN FTT see the following documentation:

What are vSAN Policies
Planning Capacity in vSAN
Demystifying Capacity Reporting in vSAN
Managing Fault Domains in vSAN Clusters
Adaptive RAID-5 Erasure Coding with the Express Storage Architecture in vSAN 8