Slow reboot of vSMP virtual machines on ESX when a lot of guest memory is page-shared

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Rebooting a virtual machine that has multiple vCPUs takes significantly longer than a single vCPU on an ESX host when using certain hardware
• Slow performance of virtual machines that use more than one vCPU on an ESX host when using certain hardware
  
  Caution: The most noticeable symptom is your virtual machine taking a significant amount of time to reboot, but it does not take significant amount of time for a fresh power on. If you are not seeing slow reboot times of virtual machines, this article does not apply to you. Do not turn off page-sharing if you are not experiencing these symptoms.

Environment

VMware vCenter Server 4.0.x
VMware vCenter Server 4.1.x
VMware ESX Server 3.0.x
VMware ESX Server 3.5.x

Resolution

This issue occurs because of changes in the architecture of certain CPUs. These changes affect the way that ESX hosts perform COW (Copy-on-Write) memory operations when using vSMP in a virtual machine.

 

There are two common ways to workaround this problem. Perform one of the following workarounds:

• Instead of rebooting the virtual machine, power it off, then power it back on. This resolves the issue because the COW memory is cleared and repopulated upon startup of the virtual machine.
• Disable page-sharing to resolve the slow performance. Page-sharing is used to reclaim duplicated memory pages across and within virtual machines. If page-sharing is disabled, virtual machines do not share memory in this way and therefore the virtual machine consumes more physical memory because all memory pages are duplicated for each virtual machine running on the ESX host.
  
  Note: Turning page-sharing off can potentially increase the amount of swapping if you are over-committing the ESX host memory resources.

To disable page-sharing on the ESX host:

1. Log in to VirtualCenter (or the ESX host directly) with an administrative account using the VMware Infrastructure (VI) Client.
2. Click on the ESX host on which you want to disable page-sharing.
3. Click the Configuration tab.
4. Click the Advanced Settings link.
5. Click Mem in the Advanced Settings window.
6. Look for the Mem.ShareScanGHz option and set the value to 0.
   
   Note: By default, Mem.ShareScanGHz is set to 4.
7. Click OK.
8. Reboot the ESX host.

If disabling page-sharing for the ESX is not an option, you can disable page-sharing for the virtual machine.

 

To disable page-sharing in a virtual machine:

1. Right-click on the virtual machine in the VI Client Inventory and choose Edit Settings.
2. Click Options and click Advanced > General.
3. Click Configuration Parameters.
4. In the dialog box that appears, click Add Row.
5. Enter sched.mem.pshare.enable and set its value to False.

Additional Information

This issue is also outlined in the known issues section of the vCenter Server release notes, for further information, see the VMware vCenter Server 4.0 Update 3 Release Notes.