Cisco UCS Servers with Intel® Optane™ DC Persistent Memory Module Support

Products

• VMware vCenter Server

Issue/Introduction

This article provides information about the persistent memory support on Cisco UCS servers.

Disclaimer:  The partner solution referenced in this article is a solution that is developed and supported by the partner. For more information, see Cisco Support and Downloads.

Resolution

Summary of server architecture, persistent memory components, and support information

Server Architecture

• Cisco UCS C240 M5 Rack Server (Small Form Factor Disk Drive Model)
• Cisco UCS C240 M5 Rack Server (Large Form Factor Disk Drive Model)
• Cisco UCS B200 M5 Blade Server
• Cisco UCS C220 M5 Rack Server (Small Form Factor Disk Drive Model)
• Cisco UCS B480 M5 Blade Server
• Cisco UCS C480 M5 Rack Server (Small form Factor Disk Drive Model)

Persistent Memory Components

Persistent Memory Type	Module Description
Intel Optane DC Persistent Memory	128 GB Intel Optane DC Persistent Memory
	256 GB intel Optane DC Persistent Memory
	512 GB Intel Optane DC Persistent Memory

Configuration Maximums

Server Model	Supported CPU Series	Certified Maximum Capacity in App-Direct Mode (TB)	Certified Maximum Capacity in Memory Mode (TB)
Cisco UCS C240 M5SX	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series  Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
Cisco UCS C240 M5L	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series  Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
Cisco UCS B200M5	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series  Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
Cisco UCS C220 M5SX	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series  Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
Cisco UCS B480M5	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series  Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
Cisco UCS C480M5	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	12	8

Notes:

• All servers that support Intel Optane DC Persistent Memory supports the following NVDIMM Platform Capabilities:
  
      •    Memory Controller Flush to NVDIMM Durability on Power Loss Capable
      
• The ACPI Specification 6.2 defines in section 5.2.20.4 the NVDIMM Platform Capabilities:
  
      1. CPU Cache Flush to NVDIMM Durability on Power Loss Capable.
      2. Memory Controller Flush to NVDIMM Durability on Power Loss Capable.
      3. Byte Addressable Persistent Memory Hardware Mirroring Capable.
      
• The certified maximum persistent memory capacity is based on installed memory capacity. There is a small difference between installed persistent memory capacity and amount available to ESXi because the system reserves some memory for internal use.  
• App-Direct Mode is supported in vSphere 6.7 U2 + Express Patch 10(ESXi670-201906002) or later.
• Memory Mode is supported in:
  •     vSphere 6.7 U2 + Express Path 10 (ESXi670-201906002) or later
    • Earlier 6.7 releases are not supported.

Additional Configuration Details

Support Information

vSphere Support for Intel’s Persistent Memory

Persistent Memory certified servers with Intel Optane persistent memory 100 series and 2nd Gen Intel Xeon Scalable processors in select configurations, as documented in https://kb.vmware.com/s/article/67645, are eligible for VMware support.

Balanced Profile optimizes Memory Mode performance by balancing the thresholds between DRAM and Intel Optane PMem to improve the bandwidth for all DRAM cache misses while maintaining the performance of DRAM cache hits. Balanced profile enabled BIOS has shown better performance in memory mode workloads.   For more information on balanced profile BIOS support, please contact your server vendor. 

Intel® Optane™ DC Persistent Memory “Memory Mode” Production Deployment Advisory

Customers interested in using Intel’s Optane DC Persistent Memory in “Memory” mode on the above-certified OEM servers with configurations not published in KB67645, https://kb.vmware.com/s/article/67645, must contact VMware to file an intent for production deployment request.

At present, production deployment of unpublished configurations requires explicit approval from VMware for support. VMware will continue to work with its customers and ecosystem to adjust the criteria for production deployments. For more information, customers should contact VMware representative.

Cisco Support Information

Cisco Minimum BIOS Versions for Balanced Profile Support

Below are minimum BIOS versions required to support the balanced profile referenced in KB67645: 

i. Cisco UCS C240 M5SX- C240M5.4.0.4j.0.0831191216
ii. Cisco UCS C240 M5L- C240M5.4.0.4j.0.0831191216
iii. Cisco UCS C220 M5SX- C220M5.4.0.4i.0.0831191119
iv. Cisco UCS C480M5- C480M5.4.0.4i.0.0831191123
v. Cisco UCS B200M5- B200M5.4.0.4i.0.0831191120
vi. Cisco UCS B480M5- B480M5.4.1.1c.0.0405200025

Troubleshooting and Best Practices 

Re-configure from 2LM mode to AppDirect mode will result in PMEM datastore not visible with below steps:

1. Configure NVDIMMS to 2 LM mode.
2. Configure NVDIMMS to App-direct mode.
3. In ESXi host execute command 'df  -h'.

Perform these steps for the issue mentioned above:

1. Boot in AppDirect mode.
2. Boot to BIOS and secure erase.
3. Save and reboot to BIOS and reconfigure to 2LM mode.
4. Boot in 2LM mode.
5. Boot to BIOS and reconfigure to AppDirect mode.
6. Boot in AppDirect mode and found PMem DS mounted with the correct size.

For more information on Cisco systems persistent memory support, see Cisco UCS: Configuring and Managing Intel® Optane™ Data Center Persistent Memory Modules.