IOMMU Configuration Restrictions on AMD EPYC (Zen 3 to Zen 5) with SEV-SNP Enabled

Products

• VMware Cloud Foundation

Issue/Introduction

When AMD SEV-SNP is enabled at the BIOS level on AMD EPYC platforms (Zen 3, Zen 4, and Zen 5), it introduces hardware limitations that restrict ESX's control over the host Input-Output Memory Management Unit (IOMMU).

Specifically, ESXi is prevented from configuring the host IOMMU in passthrough (hardware bypass) mode.

The exact impact on system resources depends on your hardware configuration, specifically your total memory capacity and processor speed.

• Memory Overhead: Minimal. Memory consumption is generally expected to remain under 1% of the total system memory.
• Boot Time Delay: Variable. For reference, testing on an AMD EPYC Zen 5 host equipped with 768GB of RAM demonstrated an approximate 7-second increase in IOMMU initialization time.

Environment

ESX 9.0 onwards

Cause

Hardware bypass modes are unavailable when SEV-SNP is active. The vmkernel's default ‘vmkDomain’ gets initialized using identity-mapped IO page tables.
Building these tables introduces slight overhead, resulting in un-noticeable increase in memory consumption and boot times, during the host IOMMU initialization phase.

Resolution

This is an expected architectural behavior on AMD Zen 3 through Zen 5 platforms when utilizing SEV-SNP security features.

No corrective action is required.

However, if you are not deploying confidential VMs, you could disable SEV-SNP in the BIOS.