Hitachi Servers with Intel® Optane™ DC Persistent Memory Module Support

Products

VMware vSphere ESXi

Issue/Introduction

This article provides information about the persistent memory support on Hitachi Servers.

Disclaimer: The partner solution referenced in this article is a solution that is developed and supported by the partner. For more information, see Hitachi HA8000V.

Environment

VMware ESXi 6.7.x
VMware vSphere ESXi 7.0.x

Resolution

Summary of server architecture, persistent memory components, and support information.

Server Architecture

Intel Optane persistent memory 100 series for second-generation Intel® Xeon® Scalable processors provides a combination of affordable large capacity and support for data persistence. Intel Optane persistent memory 100 series uses the standard DIMM form factor and is installed alongside DIMMs in a server memory slot.

Persistent Memory Components

Persistent Memory Modules

Persistent Memory Type	Module Description
Intel® Optane™ persistent Memory	Intel Optane 128GB persistent memory 100 series
	Intel Optane 256GB persistent memory 100 series
	Intel Optane 512GB persistent memory 100 series

Configuration Maximums

Server Model	Supported CPU Series	Certified Maximum Capacity in App-Direct Mode (TB)	Certified Maximum Capacity in Memory Mode (TB)
HA8000V/DL380 Gen10	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
HA8000V/DL360 Gen10	Intel Xeon Platinum 8200 (Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6

Note:

All servers that support Intel Optane DC Persistent Memory supports the following NVDIMM Platform Capabilities:
- Memory Controller Flush to NVDIMM Durability on Power Loss Capable
The ACPI Specification 6.2 defines in section 5.2.20.4 the NVDIMM Platform Capabilities:
1. CPU Cache Flush to NVDIMM Durability on Power Loss Capable
2. Memory Controller Flush to NVDIMM Durability on Power Loss Capable
3. Byte Addressable Persistent Memory Hardware Mirroring Capable
The certified maximum persistent memory capacity is based on installed memory capacity. There is a small difference between installed persistent memory capacity and amount available to ESXi because the system reserves some memory for internal use.
App-Direct Mode is supported in vSphere 6.7 U2 + Express Patch 10(ESXi670-201906002) or later.
Memory Mode is supported in
- vSphere 6.7 U2 + Express Path 10(ESXi670-201906002) or later
- Earlier 6.7 releases are not supported.

Support Information

vSphere Support for Intel Optane Persistent Memory 100 Series
Persistent Memory certified servers with Intel Optane persistent memory 100 series and 2nd Gen Intel Xeon Scalable processors in select configurations, as documented in vSphere Support for Intel's Optane Persistent Memory (PMEM), are eligible for VMware support.

Balanced Profile optimizes Memory Mode performance by balancing the thresholds between DRAM and Intel Optane PMem to improve the bandwidth for all DRAM cache misses while maintaining the performance of DRAM cache hits. Balanced profile enabled BIOS has shown better performance in memory mode workloads. For more information on balanced profile BIOS support, please contact your server vendor.

Intel® Optane™ Persistent Memory 100 Series “Memory Mode” Production Deployment Advisory

Customers interested in using Intel’s Optane persistent memory 100 series in “Memory” mode on the above-certified OEM servers with configurations not published in KB67645, https://knowledge.broadcom.com/external/article?legacyId=67645, must contact VMware to file an intent for production deployment request.

At present, production deployment of unpublished configurations requires explicit approval from VMware for support. VMware will continue to work with its customers and ecosystem to adjust the criteria for production deployments. For more information, customers should contact VMware representative.

Hitachi’s Customer Advisory

Intel Optane persistent memory 100 series ARS error handling implementation may have significant impact to PMem availability in certain failure scenarios. The first 2MB of every namespace is used by ESXi for storing PMem's critical metadata. VMKernel PMem typically expects ARS clear error DSM to succeed. Uniquely in Hitachi’s designs, ESXi is unable to clear an ARS error, and if an ARS error happens in this 2MB region, then ESXi cannot reformat and create PMem file system on the DIMMs of those namespaces. Once one namespace cannot be formatted, ESXi errors out and cannot create a PMem datastore on the host rendering all PMem in the host unusable.

The module(s) with uncorrectable error in the first 2MB of a namespace must be replaced before ESXi can create a PMem datastore. IML logs will record an event when a uncorrectable error is detected on any memory module. IML logs can be used to determine which PMem is defective and should be replaced. Part replacement could lead to significant system downtime. Contact Hitachi for additional details.

All PMem implementations are required to perform integrity checks on PMem to ensure all errors are detected and reported before the OS can be allowed access the PMem. PMem implementations differ on when and how such integrity checks are performed. Hitachi’s system BIOS will perform a PMem integrity check on boot when necessary, such as when a system reset is caused by a fatal error. Depending on the amount of PMem memory installed in an Hitachi’s System, the PMem integrity check may take a long time to complete. For example, the PMem integrity check can take 1.5 hours to complete on a 2-socket system with 6TB DCPMM.
BIOS 2.30 or later is required to support Balanced Profile referenced in vSphere Support for Intel's Optane Persistent Memory (PMEM).