Guidance for vSAN on Intel® Xeon® 6700/6500-Series Processors with P-Cores (codenamed “Granite Rapids”)

Products

• VMware vSAN

Issue/Introduction

• Broadcom will provide support for Intel® Xeon® 6700/6500-Series Processors with P-Cores (codenamed “Granite Rapids”) based Ready Nodes from OEM partners for vSAN in VCF 5.2 (vSAN 8.0 U3e) . However, performance variations have been observed in vSAN when running on Intel® Xeon® 6700/6500-Series Processors with P-Cores under certain system configurations and workload types.
• This document summarizes the performance impact when running vSAN on Intel® Xeon® 6700/6500-Series Processors with P-Cores systems in order to help customers make more informed decisions when planning vSAN deployment. Broadcom recommends that our customers do direct evaluation of Intel® Xeon® 6700/6500-Series Processors with P-Cores based vSAN Ready Nodes as individual applications may experience less impact than observed below.
• vSAN is engineered to deliver the best performance based on capabilities of the underlying CPU / hardware platform. The performance impact disclosed in this article is specific to Intel® Xeon® 6700/6500-Series Processors with P-Cores only.
• Although vSAN performance on Intel® Xeon® 6700/6500-Series Processors with P-Cores may be fine for many real-world workloads, performance may drop noticeably on high-core systems and potentially more on low-core multi-die systems compared to previous CPU generations, especially for synthetic workloads in the below specified environment.  

• For some internal vSAN qualification benchmarks, we observed that vSAN performance was lower on Intel® Xeon® 6700/6500-Series Processors with P-Cores when compared to equivalent 5th Gen Intel® Xeon® Scalable Processors (codenamed “Emerald Rapids”) for certain configurations. Real-world applications may see milder or no performance impact due to generally lower IO stress on average.

• It is also observed from in-house real application testing that workloads that are more IO bound (i.e., OLAP, data streaming) may experience more performance drop than workloads (i.e., OLTP) that are more compute bound or mixed.

Environment

• Platform: vSAN on Intel® Xeon® 6700/6500-Series Processors with P-Cores
• Software: vSAN in VCF 5.2 (vSAN 8.0 U3e)

Cause

Due to the design complexity of creating scalable processors with significantly increased core density and performance, potentially higher cross-socket and cross-NUMA node memory latencies may be observed on Intel® Xeon® 6700/6500-Series Processors with P-Cores when compared to previous smaller-core generations. This artifact was observed during Broadcom and Intel’s joint integration testing. This interacts with the vSAN software stack, resulting in performance degradation. We are jointly working to address this in a future release through software optimizations.

Resolution

Broadcom is actively working with Intel to address this issue in a future VCF release.

Recommendation

Plan for a major version upgrade from vSAN 8.0 U3e (VCF 5.2) to benefit from upcoming performance improvements.

Until software updates are available in future releases, Broadcom recommends the following for storage-intensive or IO-bound workloads on Intel® Xeon® 6700/6500-Series Processors with P-Cores, with vSAN:

• Benchmark your real-world workloads before deploying in production with HCIBench.
• For deployments with low core count processors (up to 48 cores), opting for a single-die configuration can yield superior performance compared to a multi-die setup.