The purpose of this article is to answer recuring questions and provide additional details with regard to the performance impact of mitigating RSBA-related vulnerabilities.
Q: Why are some VMs slower after upgrading to Linux kernel 5.19?
A: The Linux kernel maintainers, following advice from Intel, for CPUs with Return Stack Buffer Alternative (RSBA), changed default CPU vulnerability mitigation to Indirect Branch Restrict Speculation (IBRS). IBRS has a higher performance cost than previous default CPU vulnerability mitigations.
Q: Did VMware's July patches to vSphere cause this performance loss?
A: No, VMware's July patches mitigated RSBA-related vulnerability at the host level (preventing malicious VMs from attacking vSphere or other VMs), and with no measurable performance impact. VMware recommends installation of the July patches. The patches did not change VMware's optimization of virtualization of guest CPU mitigations. This implementation was created and optimized to minimize hypervisor overhead in 2018.
Q: Can I reconfigure my hosts or cluster to avoid performance regressions while running with Linux default CPU mitigations?
A: A VM on a newer host CPU (see list below) not enumerating RSBA can run with significantly reduced mitigation performance costs. The host must either:
Q: Does VMware issue specific recommendations for in-guest mitigation of RSBA?
A: No. VMware enables a customer to implement the security policy if their choice, including in-guest CPU vulnerability mitigation configuration. This choice is up to the customer. Different mitigations will have different performance and/or security properties. Consult with the OS vendor for further details.
Return Stack Buffer Alternative (RSBA) is a behavior of some Intel CPUs of the Skylake family. RSBA potentially enables an information leakage via an attack on Return Stack Buffer Underflow (RSBU). For concerned parties running software on vulnerable CPUs, this attack can be mitigated. Mitigation incurs a performance penalty.
In July 2022, researchers demonstrated exploitability of RSBU. Many vendors' existing mitigations of previous CPU vulnerabilities were insufficient to mitigate RSBU. Software vendors have added and/or are adding RSBU mitigation, potentially incurring performance penalties.
For VMware, mitigation of RSBA-based attacks fall into two categories:
Mitigates leakage from the hypervisor or guest VMs into a malicious guest VM. In July 2022, VMware released patches to vSphere, documented in VMSA-2022-0020, implementing hypervisor-specific mitigation at no visible performance cost.
Mitigates leakage between processes within the VM, or between the VM's kernel and user processes. In 2018, VMware implemented hypervisor-assistance of guest mitigation by virtualizing speculative execution control mechanisms. VMware optimized this assistance to minimize hypervisor overhead. Within a VM, a guest operating systems may implement whatever mitigation it chooses (or none at all) and vSphere will execute this faithfully.
Linux kernel 5.19 implements a new default mitigation for CPU vulnerabilities. If RSBA is detected in the underlying CPU, RSBU is mitigated using IBRS. IBRS is more expensive than the previous default mitigation (retpoline), resulting in performance loss. This performance loss is specific to workload and underlying physical CPU. Performance loss is experienced whether the underlying platform is physical (bare metal) or virtual (running on a hypervisor such as vSphere).
Windows already mitigated existing CPU vulnerabilities using IBRS by default, so there is no new overhead for RSBU mitigation in Windows guests.
Please refer to the VMware Performance Blog for additional information.
RSBA is advertised to a guest VM unless either of these following:
VMware's patches in 2018 implemented this RSBA behavior. This was not changed by VMware's July patches.
VMware enables customers to implement their choice of security policies, within their virtual machines. VMware enables these mitigation options with minimal hypervisor overheads. Customers may choose whether and how to mitigate given their policies. Operating system providers may provide mitigation alternatives with different security and performance properties.
A heterogeneous cluster of hosts (vulnerable and invulnerable) with a Skylake or older EVC mode, will advertise RSBA to all VMs therein. Splitting this cluster into different clusters (one vulnerable, one invulnerable) would allow the invulnerable cluster's VMs to run without RSBA advertised, recouping performance for those VMs.
AMD CPUs are affected by a similar vulnerability called Branch Type Confusion (BTC). Default Linux mitigation of BTC on AMD uses different techniques than the above, with a different performance footprint. These defaults were added to the Linux kernel before version 5.19. As AMD does not have RSBA/RSBU but instead BTC, the kernel affecting the change was different and the performance effects are different, this is omitted from broader discussion.