Hewlett Packard Enterprise Servers with Intel® Optane™ Persistent Memory 100 Series for HPE Support

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides information about the persistent memory support on Hewlett Packard Enterprise Servers.

Disclaimer:  The partner solution referenced in this article is a solution that is developed and supported by the partner. For more information, see https://www.hpe.com/info/persistentmemory

Environment

VMware vSphere 6.5.x
VMware vSphere 6.7.x
VMware vSphere 7.0.x

Resolution

Here is a summary of server architecture, persistent memory components, and support information.

Server Architecture

Intel Optane persistent memory 100 series for HPE for second-generation Intel® Xeon® Scalable processors provides a combination of affordable large capacity and support for data persistence.  Intel Optane persistent memory 100 series for HPE uses the standard DIMM form factor and is installed alongside DIMMs in a server memory slot.

Persistent Memory Components

Persistent Memory Modules
 

Persistent Memory Type	Module Description
Intel® Optane™ persistent Memory	Intel Optane 128GB persistent memory 100 series for HPE
Intel® Optane™ persistent Memory	Intel Optane 256GB persistent memory 100 series for HPE
Intel® Optane™ persistent Memory	Intel Optane 512GB persistent memory 100 series for HPE

Note: For NVDIMM-N support, refer to Hewlett Packard Enterprise Servers Persistent Memory (PMem) Support.

Configuration Maximums
 

Server Model	Supported CPU Series	Certified Maximum Capacity in App-Direct Mode (TB)	Certified Maximum Capacity in Memory Mode (TB)
ProLiant DL380 Gen10	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
ProLiant DL360 Gen10	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
ProLiant DL560 Gen10	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	12	7.8
ProLiant DL580 Gen10	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	12	7.8
Synergy 480 Gen10 Compute Module	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
Synergy 660 Gen10 Compute Module	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	12	7.8
ProLiant XL170r Gen10	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	1	1
ProLiant XL190r Gen10	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	1	1
ProLiant XL420 Gen10	Intel Xeon Platinum 8200 (Cascade Lake-SP)Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	1	1
ProLiant XL450 Gen10	Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	1	1

Notes:

• All servers that support Intel Optane DC Persistent Memory support the following NVDIMM Platform Capabilities:
  1. Memory Controller Flush to NVDIMM Durability on Power Loss Capable
• The ACPI Specification 6.2 defines in section 5.2.20.4 the NVDIMM Platform Capabilities: 
  1. CPU Cache Flush to NVDIMM Durability on Power Loss Capable
  2. Memory Controller Flush to NVDIMM Durability on Power Loss Capable
  3. Byte Addressable Persistent Memory Hardware Mirroring Capable

• The certified maximum persistent memory capacity is based on installed memory capacity.  There is a small difference between installed persistent memory capacity and amount available to ESXi because the system reserves some memory for internal use. 
• App-Direct Mode is supported in vSphere 6.7 U2 + Express Patch 10(ESXi670-201906002) or later.
• Memory Mode is supported in
  • vSphere 6.5 U3 or later
  • vSphere 6.7 U2 + Express Patch 10(ESXi670-201906002) or later
    • Earlier 6.7 releases are not supported.

Support Information

vSphere Support for Intel Optane Persistent Memory 100 Series

Persistent Memory certified servers with Intel Optane persistent memory 100 series and 2nd Gen Intel Xeon Scalable processors in select configurations, as documented in https://knowledge.broadcom.com/external/article?articleNumber=326910 are eligible for VMware support. 

Balanced Profile optimizes Memory Mode performance by balancing the thresholds between DRAM and Intel Optane PMem to improve the bandwidth for all DRAM cache misses while maintaining the performance of DRAM cache hits. Balanced profile enabled BIOS has shown better performance in memory mode workloads.   For more information on balanced profile BIOS support, please contact your server vendor.  

Intel® Optane™ Persistent Memory 100 Series “Memory Mode” Production Deployment Advisory

Customers interested in using Intel’s Optane persistent memory 100 series in “Memory” mode on the above-certified OEM servers with configurations not published in https://knowledge.broadcom.com/external/article?articleNumber=326910, must contact VMware to file an intent for production deployment request. 

At present, production deployment of unpublished configurations requires explicit approval from VMware for support. VMware will continue to work with its customers and ecosystem to adjust the criteria for production deployments. For more information, customers should contact VMware representative.

HPE Customer Advisory

1. Intel Optane persistent memory 100 series for HPE ARS error handling implementation may have significant impact to PMem availability in certain failure scenarios.  The first 2MB of every namespace is used by ESXi for storing PMem's critical metadata.  VMKernel PMem typically expects ARS clear error DSM to succeed.  Uniquely in HPE's designs, ESXi is unable to clear an ARS error, and if an ARS error happens in this 2MB region, then ESXi cannot reformat and create PMem file system on the DIMMs of those namespaces. Once one namespace cannot be formatted, ESXi errors out and cannot create a PMem datastore on the host rendering all PMem in the host unusable.
   
   The module(s) with uncorrectable error in the first 2MB of a namespace must be replaced before ESXi can create a PMem datastore.  HPE IML logs will record an event when a uncorrectable error is detected on any memory module.  IML logs can be used to determine which PMem is defective and should be replaced.  Part replacement could lead to significant system downtime.  Contact HPE for additional details.

2. All PMem implementations are required to perform integrity checks on PMem to ensure all errors are detected and reported before the OS can be allowed access the PMem.  PMem implementations differ on when and how such integrity checks are performed.  HPE system BIOS will perform a PMem integrity check on boot when necessary, such as when a system reset is caused by a fatal error.  Depending on the amount of PMem memory installed in an HPE System, the PMem integrity check may take a long time to complete. For example, the PMem integrity check can take 1.5 hours to complete on a 2-socket system with 6TB DCPMM.
    
3. HPE BIOS 2.30 or later is required to support Balanced Profile referenced in https://knowledge.broadcom.com/external/article?articleNumber=326910

For more information on Hewlett Packard Enterprise persistent memory support, see https://www.hpe.com/info/persistentmemory.