Disk goes into Read-Only Mode if the CPU core is increased beyond 128 for Linux Virtual Machines.

Products

• VMware vSphere ESXi

Issue/Introduction

Disks of linux virtual machines unexpectedly enter read-only mode when exceeding 128 cores, whereas Windows VM above CPU core 128 works fine.

Environment

VMware vCenter Server 7.0
VMware vCenter Server 8.0

Cause

Increasing the CPU core count above 128 for Linux virtual machines automatically enables IOMMU. This, however, requires additional configuration at both the VM level (VMX file) and the kernel level (GRUB menu) to prevent the disk from entering read-only mode. These configurations ensure proper communication between the virtual machine and the hardware, especially when dealing with high CPU core counts and potentially demanding hardware resources.

Resolution

1. To address the read-only disk issue after increasing CPU cores beyond 128, ensure the following parameters are present in the Virtual Machine's VMX configuration file:

• Power off the virtual machine.
• Open the VMX configuration file.
• Add the necessary parameters to the file.
• Save the changes and power on the virtual machine.

amd.iommu.supportsPcip=”TRUE”
vvtd.enable=“TRUE”
pciPassthru.vmiop.allowViommu=”TRUE”
pciPassthru.vmiop.enableViommu=”TRUE”
iommu=pt

2. During the virtual machine boot process, access the GRUB menu.
3. Locate the /etc/default/grub file and identify the GRUB_CMDLINE_LINUX parameter
4. Add the parameters to enable both IOMMU and passthrough at the kernel level.

GRUB_CMDLINE_LINUX= "amd_iommu=on iommu=pt"
GRUB_CMDLINE_LINUX= "iommu=soft"

 

Additional Information

Reloading a vmx file without removing the virtual machine from inventory