Linux Guest OS may become unresponsive after hot-adding memory from under 3 GB to over 3 GB

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Virtual machines installed with the some 64-bit Linux guest operating systems (RHEL 5, SLES 11) may become unresponsive if the virtual machine is started with an IDE device attached, and if memory is hot-added from below 3 GB to above 3 GB.
• If you hot-add memory to the virtual machine, you see the following messages within the virtual machine:
  
  nommu_map_sg: overflow 137c1e000+4096 of device mask ffffffff
  nommu_map_sg: overflow 137c1e000+4096 of device mask ffffffff
  nommu_map_sg: overflow 137c1e000+4096 of device mask ffffffff
  
• This issue does not occur with 32-bit operating systems.
• This issue does not occur if the virtual machine is created with >4 GB memory and more memory is hot-added.

Resolution

When the initial virtual machine memory is less than 3072 MB, the Linux OS kernel boots without enabling swiotlb. When you hot-add memory to such a kernel and increase beyond this limit, swiotlb remains disabled.

To workaround this issue, you must enable swiotlb.

To enable swiotlb:

1. If the virtual machine is configured with < 3 GB memory, try this option to the kernel at boot time:<br>
   iommu=soft
   
   If this did not work the first time, try:
   
   swiotlb=force
   
   
2. When the system boots, verify by executing the command:
   
   dmesg | grep SWIOTLB
   
   You see output similar to:
   
   PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
   

Do not use memory hot-add to change the virtual machine size from less than or equal to 3072 MB to greater than 3072 MB. Instead, power off the virtual machine to perform this reconfiguration. For more information on hot-add memory in a Linux virtual machine and bringing it online, see Hot adding memory in Linux (1012764).

Notes:

• This problem occurs only when the 3 GB mark is crossed while the operating system is running.
• If the guest OS is already unresponsive, restart the virtual machine.

Additional Information

Hot adding memory in Linux