This article documents the additional Transparent Page Sharing (TPS) management capabilities that are introduced in the following releases:
• ESXi 5.0 Patch ESXi500-201502001, released on February 26, 2015, see the VMware ESXi 5.0, Patch Release ESXi500-201502001 (2101910) for details.
• ESXi 5.1 Update 3 released on December 4, 2014, see the VMware ESXi 5.1 Update 3 Release Notes for details.
• ESXi 5.5, Patch ESXi550-201501001, released on January 27, 2015, see the VMware ESXi 5.5, Patch Release ESXi550-201501001 (2099265) for details.
• ESXi 6.0
(See the Frequently Asked Questions section below for the additional Transparent Page Sharing (TPS) management capabilities introduced in earlier ESXi patch releases)
This KB also explains the concept of salting and the corresponding configuration options.
Please refer to Security considerations and disallowing inter-Virtual Machine Transparent Page Sharing (2080735) KB for the background on the changes to the default TPS setting in the ESXi Update releases listed above. These changes are related to recent academic research that leverages Transparent Page Sharing (TPS) to gain unauthorized access to data under certain highly controlled conditions.
The concept of salting has been introduced to help address concerns system administrators may have over the security implications of TPS as described in KB Security considerations and disallowing inter-Virtual Machine Transparent Page Sharing (2080735). Salting is used to allow more granular management of the virtual machines participating in TPS than was previously possible. As per the original TPS implementation, multiple virtual machines could share pages when the contents of the pages were same. With the new salting settings, the virtual machines can share pages only if the salt value and contents of the pages are identical. A new host config option Mem.ShareForceSalting is introduced to enable or disable salting.
By default, salting is enabled after the ESXi update releases mentioned above are deployed, (Mem.ShareForceSalting=2) and each virtual machine has a different salt. This means page sharing does not occur across the virtual machines (inter-VM TPS) and only happens inside a virtual machine (intra VM).
When salting is enabled (Mem.ShareForceSalting=1 or 2) in order to share a page between two virtual machines both salt and the content of the page must be same. A salt value is a configurable vmx option for each virtual machine. You can manually specify the salt values in the virtual machine's vmx file with the new vmx option sched.mem.pshare.salt. If this option is not present in the virtual machine's vmx file, then the value of vc.uuid vmx option is taken as the default value. Since the vc.uuid is unique to each virtual machine, by default TPS happens only among the pages belonging to a particular virtual machine (Intra-VM).
If a group of virtual machines are considered trustworthy, it is possible to share pages among them by setting a common salt value for all those virtual machines (inter-VM).
The following table shows how different settings for TPS are used together to effect how TPS operates for individual virtual machines:
Mem. ShareForceSalting (host setting) |
sched.mem.pshare.salt (per VM setting) |
vc.uuid (per VM setting) |
Salt value of VM |
TPS between VMs (Inter-VM) |
TPS within a VM (Intra-VM) |
0 |
Ignored |
Ignored |
0 |
Yes, among all VMs on host. |
yes |
1 |
Present |
Ignored |
sched.mem.pshare.salt |
Only among VMs with same salt |
yes |
1 |
Not Present |
Ignored |
0 |
Yes, among all VMs |
yes |
2 |
Present |
Ignored |
sched.mem.pshare.salt |
Only among VMs with same salt |
yes |
2 |
Not Present |
Present (default) |
vc.uuid |
No inter-VM TPS |
yes |
2 |
Not Present |
Not Present |
random number |
No inter-VM TPS |
yes |
For more information on TPS, refer the following blog:
What is meant by Intra-VM and Inter-VM in the context of Transparent Page Sharing?
Where can I find more information on Transparent Page Sharing?
For more information on TPS, see:
What is the default behavior of Transparent Page Sharing in above mentioned Update releases?
By default, after deploying the ESXi Update releases mentioned above salting is enabled (Mem.ShareForceSalting=2) and each virtual machine has a different salt (that is sched.mem.pshare.salt is not present) which means that only Intra-VM page sharing is enabled. This behavior is new as per these ESXi update releases and page sharing will not happen across the virtual machines (inter-VM TPS) by default.
How do I re-enable inter-VM TPS for all virtual machines after deploying an ESX Update release that no longer has inter-VM TPS enabled by default?
Set MEM_SHARE_FORCE_SALTING to 0.
How do I re-enable inter-VM TPS for selected virtual machines after deploying an ESX Update release that no longer has inter-VM TPS enabled by default?
Set MEM_SHARE_FORCE_SALTING to 1 or 2 and for the virtual machines you wish to share, set sched.mem.pshare.salt to a common value.
How can I enable or disable salting?
Set advanced memory config option as ShareForceSalting. Follow these steps to enable or disable salting:
How can I allow inter-VM TPS between two or more virtual machines?
Inter-VM TPS is enabled for two or more virtual machines by enabling salting and by giving them the same salt value.
How can I specify salt value of a virtual machine?
Steps to specify the salt value for a virtual machine:
Note: Same salting values can be specified to achieve the page sharing across virtual machines.
What is the difference in behavior of page sharing when MEM_SHARE_FORCE_SALTING value is set to 1 and 2?
MEM_SHARE_FORCE_SALTING 1: By default salt value is taken from sched.mem.pshare.salt. If not specified, falls back to old TPS (inter-VM) behavior by considering salt values for the virtual machine as 0.
MEM_SHARE_FORCE_SALTING 2: By default salt value is taken from vc.uuidz. If it does not exist, then the page sharing algorithm generates random and unique value for salting per virtual machine, which is not configurable by users.
<?xml:namespace prefix = "o" />
How to manage the additional Transparent Page Sharing (TPS) management capabilities introduced in the earlier TPS ESXi patches?
The following patch releases have introduced additional Transparent Page Sharing (TPS) management capabilities for the first time. KB Additional Transparent Page Sharing management capabilities in ESXi 5.5, 5.1, and 5.0 patches in Q4, 2014 (2091682) explains how to manage these capabilities when the patches are deployed and when the Update releases listed on the top of this KB are not deployed:
How can I prepare for the ESXi Update releases that no longer allow inter-VM TPS by default?
VMware recommends you to monitor free memory available on the host along with the total ballooned and total swapped memory before deploying the ESXi update releases listed above that disallow inter-VM TPS. Once inter-VM TPS is disallowed, available free memory might drop which further can lead to increased ballooning and swapping. If increased ballooning and swapping activity is observed along with noticeable performance issues, more physical memory can be added on the host or the memory load on the host can be reduced.
To monitor the stats - Run esxtop(1) command:
See also the VMware vSphere Blog entry Assess the Performance Impact of the Security Change in Transparent Page Sharing Behaviour .
How can I enable or disable salting for multiple ESXi hosts?
To enable or disable salting for multiple ESXi hosts. Refer to the attached powercli script. This script allows toggling pshare salting for update releases.
Usage
.\pshare-salting.ps1 <vcenter IP/hostname> -s -> Enables pshare salting.
.\pshare-salting.ps1 <vcenter IP/hostname> -o -> Turn offs pshare salting and falls back to default TPS behavior.