Memory Tiering Tier 1 Capacity shows the same size as DRAM Size

Products

• VMware Cloud Foundation
• VMware vCenter Server

Issue/Introduction

• After configuring Memory Tiering with an NVMe device, the Tier 1 capacity in the vSphere Client shows only a fraction of the total NVMe disk capacity.
  
  
• Tier 1 capacity is showing the same size of the host DRAM, even with large sized NVMe used for Memory tiering.
  
  
• In vCenter UI, Host > Summary shows the size of the Tier 1 capacity same as Tier 0 (DRAM) size.
  
  

Environment

• VMware Cloud Foundation (VCF) 9.1
• Memory Tiering over NVMe feature

Cause

• This is the default architectural behavior of the Memory Tiering feature. When Memory Tiering is enabled, the system by default configures the Tier 1 (NVMe) capacity to equal 100% of the physical DRAM (Tier 0) size.

• From Host CLI as we could see the Memory Tiering is enabled on the host and by default configured with 100% of DRAM size.

  # esxcli memtier status get
   Status: ENABLED
   Devices: ####
   Size: 511.68 GiBs (100 percent of DRAM)
   Encryption: Disabled
   License Info: Licensed for 4.00 times of DRAM

Resolution

This is an expected behavior.

Depending on your workloads, the Tier 1 capacity can be increased upto 4 times of the DRAM. 

• Steps to update the size of the Tier 1 from the vCenter UI.

1. 1. From vCenter UI, Click on the Host > Configure.
   2. Select Advanced System Settings.
   3. Select the Key 'Mem.TierNVMePct'
   4. Update the Percentage of DRAM to use as NVMe tier (Value between 1-400)

• Command to update the memtier percentage from Host CLI.

# esxcli memtier config set --tier-size-pct <Value between 1-400>

Note:
The recommendation and best practice advice is to keep the default (100 percent of DRAM) as it is applicable to most workloads. Before changing the default ratio, first thoroughly evaluate the workloads and make sure all active memory can fit in the host’s DRAM capacity.