NEC High Performance Servers with Intel® Optane™ Persistent Memory 100 Series Support

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides information about the persistent memory support on NEC High Performance Servers.

Disclaimer:

The partner solution referenced in this article is a solution that is developed and supported by the partner. For more information, see<https://www.nec.com/en/global/prod/express/related/vmware/persistent_memory/index.html>.

Environment

VMware vSphere ESXi 6.7

Resolution

Here is a summary of server architecture, persistent memory components, and support information.

Server Architecture

Intel Optane persistent memory 100 seriesfor NEC for second-generation Intel® Xeon® Scalable processors provides a combination of affordable large capacity and support for data persistence.  Intel Optane persistent memory100 seriesforNEC uses the standard DIMM form factor and is installed alongside DIMMs in a server memory slot.

Persistent Memory Components

Persistent Memory Modules
 

Persistent Memory Type	Module Description
Intel Optane Persistent Memory	128GB Intel Optane Persistent Memory 100 Series
	256GB Intel Optane Persistent Memory 100 Series
	512GB Intel Optane Persistent Memory 100 Series

Configuration Maximums
 

Server Model	Supported CPU Series	Certified Maximum Capacity in App-Direct Mode(TB)	Certified Maximum Capacity in Memory Mode (TB)
Express5800/R120h-2M	Intel Xeon Platinum 8200(Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6
Express5800/R120h-1M	Intel Xeon Platinum 8200(Cascade Lake-SP) Series Intel Xeon Gold 6200/5200 (Cascade Lake-SP) Series	6	6

Note:

• All servers that support Intel Optane Persistent Memory 100 Series supports the following NVDIMM Platform Capabilities:
  • Memory Controller Flush to NVDIMM Durability on Power Loss Capable
• The ACPI Specification 6.2 defines in section 5.2.20.4 the NVDIMM Platform Capabilities: 
  1. CPU Cache Flush to NVDIMM Durability on Power Loss Capable
  2. Memory Controller Flush to NVDIMM Durability on Power Loss Capable
  3. Byte Addressable Persistent Memory Hardware Mirroring Capable
• The certified maximum persistent memory capacity is based on installed memory capacity.  There is a small difference between installed persistent memory capacity and amount available to ESXi because the system reserves some memory for internal use. 
• App-Direct Mode is supported in vSphere 6.7 U2 + Express Patch 10(ESXi670-201906002) or later.
• Memory Mode is supported in
  • vSphere 6.5 U3 or later
  • vSphere 6.7 U2 + Express Path 10(ESXi670-201906002) or later
    • Earlier 6.7 releases are not supported.

Additional Configuration Details:

Support Information

Sphere Support for Intel Optane Persistent Memory 100 Series

Persistent Memory certified servers with Intel Optane persistent memory 100 series and 2nd Gen Intel Xeon Scalable processors in select configurations, as documented in vSphere Support for Intel's Optane Persistent Memory (PMEM) , are eligible for VMware support.

Balanced Profile optimizes Memory Mode performance by balancing the thresholds between DRAM and Intel Optane PMem to improve the bandwidth for all DRAM cache misses while maintaining the performance of DRAM cache hits. Balanced profile enabled BIOS has shown better performance in memory mode workloads.   For more information on balanced profile BIOS support, please contact your server vendor. 

Intel® Optane™ Persistent Memory 100 Series “Memory Mode” Production Deployment Advisory

Customers interested in using Intel’s Optane persistent memory 100 series in “Memory Mode” on the above-certified OEM servers with configurations not published in vSphere Support for Intel's Optane Persistent Memory (PMEM), contact VMware to file an intent for production deployment request.

At present, production deployment of unpublished configurations requires explicit approval from VMware for support. VMware will continue to work with its customers and ecosystem to adjust the criteria for production deployments. For more information, customers should contact VMware representative.

NEC Customer Advisory

1. Intel Optane persistent memory 100 series for NEC ARS error handling implementation may have significant impact to PMem availability in certain failure scenarios.  The first 2MB of every namespace is used by ESXi for storing PMem's critical metadata.  VMKernel PMem typically expects ARS clear error DSM to succeed.  Uniquely in NEC's designs, ESXi is unable to clear an ARS error, and if an ARS error happens in this 2MB region, then ESXi cannot reformat and create PMem file system on the DIMMs of those namespaces. Once one namespace cannot be formatted, ESXi errors out and cannot create a PMem datastore on the host rendering all PMem in the host unusable.
   The module(s) with uncorrectable error in the first 2MB of a namespace must be replaced before ESXi can create a PMem datastore.  NEC IML logs will record an event when a uncorrectable error is detected on any memory module.  IML logs can be used to determine which PMem is defective and should be replaced.  Part replacement could lead to significant system downtime.  Contact NEC for additional details.
    
2. All PMem implementations are required to perform integrity checks on PMem to ensure all errors are detected and reported before the OS can be allowed access the PMem.  PMem implementations differ on when and how such integrity checks are performed.  NEC system BIOS will perform a PMem integrity check on boot when necessary, such as when a system reset is caused by a fatal error.  Depending on the amount of PMem memory installed in an NEC System, the PMem integrity check may take a long time to complete. For example, the PMem integrity check can take 1.5 hours to complete on a 2-socket system with 6TB DCPMM. 
    
3. NEC BIOS 2.30 or later is required to support Balanced Profile referenced in vSphere Support for Intel's Optane Persistent Memory (PMEM).

For more information on NEC persistent memory support, see NEC Persistent Memory.