Additional Transparent Page Sharing management capabilities and new default settings
Article ID: 323624
Updated On:
Products
Issue/Introduction
This article documents the additional Transparent Page Sharing (TPS) management capabilities that are introduced in ESXi 5.x and above.
(See the Legacy Release Archives section in VMware Docs for more details)
This KB also explains the concept of salting and the corresponding configuration options.
Please refer to Security considerations and disallowing inter-Virtual Machine Transparent Page Sharing (2080735) KB for the background on the changes to the default TPS setting in the ESXi Update releases listed above. These changes are related to recent academic research that leverages Transparent Page Sharing (TPS) to gain unauthorized access to data under certain highly controlled conditions.
Environment
VMware vSphere ESXi 8.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 6.x
VMware vSphere ESXi 5.x
Resolution
The concept of salting has been introduced to help address concerns system administrators may have over the security implications of TPS as described in KB Security considerations and disallowing inter-Virtual Machine Transparent Page Sharing (2080735). Salting is used to allow more granular management of the virtual machines participating in TPS than was previously possible. As per the original TPS implementation, multiple virtual machines could share pages when the contents of the pages were same. With the new salting settings, the virtual machines can share pages only if the salt value and contents of the pages are identical. A new host config option Mem.ShareForceSalting is introduced to enable or disable salting.
By default, salting is enabled after the ESXi update releases mentioned above are deployed, (Mem.ShareForceSalting=2) and each virtual machine has a different salt. This means page sharing does not occur across the virtual machines (inter-VM TPS) and only happens inside a virtual machine (intra VM).
When salting is enabled (Mem.ShareForceSalting=1 or 2) in order to share a page between two virtual machines both salt and the content of the page must be same. A salt value is a configurable vmx option for each virtual machine. You can manually specify the salt values in the virtual machine's vmx file with the new vmx option sched.mem.pshare.salt. If this option is not present in the virtual machine's vmx file, then the value of vc.uuid vmx option is taken as the default value. Since the vc.uuid is unique to each virtual machine, by default TPS happens only among the pages belonging to a particular virtual machine (Intra-VM).
If a group of virtual machines are considered trustworthy, it is possible to share pages among them by setting a common salt value for all those virtual machines (inter-VM).
The following table shows how different settings for TPS are used together to effect how TPS operates for individual virtual machines:
Mem. ShareForceSalting (host setting) |
sched.mem.pshare.salt (per VM setting) |
vc.uuid (per VM setting) |
Salt value of VM |
TPS between VMs (Inter-VM) |
TPS within a VM (Intra-VM) |
|
0 |
Ignored |
Ignored |
0 |
Yes, among all VMs on host. |
yes |
|
1 |
Present |
Ignored |
sched.mem.pshare.salt |
Only among VMs with same salt |
yes |
|
1 |
Not Present |
Ignored |
0 |
Yes, among all VMs |
yes |
|
2 |
Present |
Ignored |
sched.mem.pshare.salt |
Only among VMs with same salt |
yes |
|
2 |
Not Present |
Present (default) |
vc.uuid |
No inter-VM TPS |
yes |
|
2 |
Not Present |
Not Present |
random number |
No inter-VM TPS |
yes |
For more information on TPS, refer the following blog:
Frequently Asked Questions
What is meant by Intra-VM and Inter-VM in the context of Transparent Page Sharing?
- Intra-VM means that TPS will de-duplicate identical pages of memory within a virtual machine, but will not share the pages with any other virtual machines.
- Inter-VM mean that TPS will de-duplicate identical pages of memory within a virtual machine and will also share the duplicates with one of more other virtual machines with the same content.
What is the default behavior of Transparent Page Sharing in above mentioned Update releases?
By default, after deploying the ESXi Update releases mentioned above salting is enabled (Mem.ShareForceSalting=2) and each virtual machine has a different salt (that is sched.mem.pshare.salt is not present) which means that only Intra-VM page sharing is enabled. This behavior is new as per these ESXi update releases and page sharing will not happen across the virtual machines (inter-VM TPS) by default.
How do I re-enable inter-VM TPS for all virtual machines after deploying an ESX Update release that no longer has inter-VM TPS enabled by default?
Set MEM_SHARE_FORCE_SALTING to 0.
How do I re-enable inter-VM TPS for selected virtual machines after deploying an ESX Update release that no longer has inter-VM TPS enabled by default?
Set MEM_SHARE_FORCE_SALTING to 1 or 2 and for the virtual machines you wish to share, set sched.mem.pshare.salt to a common value.
How can I enable or disable salting?
Set advanced memory config option as ShareForceSalting. Follow these steps to enable or disable salting:
- Log in to ESX (i)/vCenter with the VI-Client.
- Select ESX (i) relevant host.
- In the Configuration tab, click Advanced Settings (link) under the software section.
- In the Advanced Settings window, click Mem.
- Search for Mem.ShareForceSalting and set the value to 1 or 2(enable salting), 0(disable salting).
- Click OK.
- For the changes to take effect do either of the two:
- Migrate all the virtual machines to another host in cluster and then back to original host. Or
- Shutdown and power-on the virtual machines.
How can I allow inter-VM TPS between two or more virtual machines?
Inter-VM TPS is enabled for two or more virtual machines by enabling salting and by giving them the same salt value.
How can I specify salt value of a virtual machine?
Steps to specify the salt value for a virtual machine:
- Power off the virtual machine on which you want to set salt value.
- Right click on virtual machine, click on Edit settings.
- Select options menu, click on General under Advanced section.
- Click on Configuration Parameters….
- Click on Add Row, new row will be added.
- On LHS add text sched.mem.pshare.salt and on RHS specify the unique string.
- Power on the virtual machine to take effect of salting.
- Repeat steps 1 to 7 to set the salt value for individuals virtual machine.
Note: Same salting values can be specified to achieve the page sharing across virtual machines.
What is the difference in behavior of page sharing when MEM_SHARE_FORCE_SALTING value is set to 1 and 2?
MEM_SHARE_FORCE_SALTING 1: By default salt value is taken from sched.mem.pshare.salt. If not specified, falls back to old TPS (inter-VM) behavior by considering salt values for the virtual machine as 0.
MEM_SHARE_FORCE_SALTING 2: By default salt value is taken from vc.uuidz. If it does not exist, then the page sharing algorithm generates random and unique value for salting per virtual machine, which is not configurable by users.
How can I prepare for the ESXi Update releases that no longer allow inter-VM TPS by default?
VMware recommends you to monitor free memory available on the host along with the total ballooned and total swapped memory before deploying the ESXi update releases listed above that disallow inter-VM TPS. Once inter-VM TPS is disallowed, available free memory might drop which further can lead to increased ballooning and swapping. If increased ballooning and swapping activity is observed along with noticeable performance issues, more physical memory can be added on the host or the memory load on the host can be reduced.
To monitor the stats - Run esxtop(1) command:
- Run esxtop on host, click m to switch to memory mode.
- free from PMEM /MB row displays the free memory available on the host.
- curr from MEMCTL/MB row displays the total ballooned memory.
- curr from SWAP/MB row displays the total swapped memory.
See also the VMware vSphere Blog entry Assess the Performance Impact of the Security Change in Transparent Page Sharing Behaviour .
How can I enable or disable salting for multiple ESXi hosts?
To enable or disable salting for multiple ESXi hosts. Refer to the attached powercli script. This script allows toggling pshare salting for update releases.
Usage
.\pshare-salting.ps1 <vcenter IP/hostname> -s -> Enables pshare salting.
.\pshare-salting.ps1 <vcenter IP/hostname> -o -> Turn offs pshare salting and falls back to default TPS behavior.
Additional Information
For translated versions of this article, see:
Attachments
Feedback
