Installing and upgrading the latest version of VMware Tools on existing hosts

Products

• VMware vSphere ESXi

Issue/Introduction

Besides the use of VMware ESXi installer and VMware vSphere Update Manager (VUM) or VMware vSphere Lifecycle Manager (vLCM) to manage the version of VMware Tools ISO image bundle available on VMware ESXi hosts, VMware Tools ISO images files can be installed and updated manually.

For more information about upgrading VMware Tools using Update Manager on vSphere 6.7, see Upgrade VMware Tools for Virtual Machines
For more information about upgrading VMware Tools using vSphere Lifecycle Manager on vSphere 7.0, see Upgrade the VMware Tools Version of Virtual Machines

This article provides:

• Manual steps to install or upgrade to the latest version of VMware Tools ISO image files for existing VMware ESXi Hosts.
• Manual steps to complete the install or upgrade without rebooting the VMware ESXi hosts.

This article also helps in resolving the errors or issues:

• The required VMware Tools ISO image does not exist or is inaccessible. Vix 21001
• VMware Tools reported as “up to date” after updating the VMware ESXi hosts VMware Tools ISO image files to a newer version.

Environment

VMware vSphere ESXi 7.0.x
VMware vSphere ESXi 6.0
VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.7

Resolution

The following approaches are supported to install and upgrade to the latest version of VMware Tools independently of the VMware ESXi version.

Installing or updating VMware Tools default repository on the VMware ESXi Host

The VMware ESXi installer, VUM, and vLCM always install VMware Tools ISO image files in the default /locker partition location (/locker/packages/vmtoolsRepo) and are not covered by this article.

It is not recommended to manually update the content of the default VMware Tools ISO image files repository found in the /locker partition (/locker/packages/vmtoolsRepo). Doing so will conflict with the VMware ESXi installer, VUM, and vCLM operations managing the content.

Note:
The ProductLocker location (/productLocker symbolic link) is controlled by the UserVars.ProductLockerLocation setting. The setting can be found on the ESXi Host’s Web UI: host -> manage -> Advanced Settings, or on VMware vCenter Web UI: Host -> Configure -> Advanced System Settings; then search for UserVars.ProductLockerLocation. If not set, the /productLocker symbolic link normally points to /locker/packages/vmtoolsRepo, and VMware Tools ISO image files from that location are in use for VMware Tools installation and upgrade. The /productLocker symbolic link can point to /tools when the ramdisk cache is in use.
 

Issue:
After installing or updating the VMware Tools repository on the VMware ESXi hosts, the VMware Tools version status of the existing virtual machines does not indicate a new version of VMware Tools is available for install or upgrade.

Note:
The interval at which the VMware Tools repository is monitored for updates is controlled by the Config.HostAgent.plugins.vmsvc.productLockerWatchInterval (in seconds). The settings defaults to 300s; and is accessible through the Web UI’s host “Advanced Settings” (VMware ESXi) or “Advanced System Setting” (VMware vCenter) lists.

If the issue persists past the VMware Tools monitoring interval, this might be due to the VMware ESXi host caching the VMware Tools repository in a RAM disk (/tools ramdisk)

1. First check if the /tools ramdisk is in use through ESXi Host CLI . (see Connecting to an ESX host using an SSH client )

# ls -l /productLocker

2. If the output of the command above shows “/tools” as the symbolic link target. The content of the /tools ramdisk can be updated by doing one of
   1. Rebooting the VMWare ESXi host, or
   2. Copying the content of /locker/packages/vmtoolsRepo/ to /tools with:

# rm -rf /tools/*
# cp -r /locker/packages/vmtoolsRepo/* /tools/

This can also be due to an incorrect configuration for the ProductLocker location discussed in the next section.

Installing or updating VMware Tools repository on a datastore

The VMware Tools repository is (or will be) located on a (shared) datastore accessible to the VMware ESXi Hosts.

Before you begin

When using a datastore path, ProductLocker should be configured before powering on VMs on the host. Installing or upgrading VMware Tools might fail for VMs powered on before configuring ProductLocker. For more information, see Installing or upgrading VMware Tools might fail for VMs powered on before configuring ProductLocker (2147383).

When installing VMware Tools repository on a datastore for the first time, you first need to create a directory to use as the root of the repository.

The datastore location for the repository and the directory:

• Must be accessible to the VMware ESXi Hosts
• Must be only for use as the VMware Tools ISO images repository (i.e.: not a Virtual Machine folder or home)
• Create the VMware Tools repository on the datastore, and adjust the permissions:

# mkdir /vmfs/volumes/<datastore-name-or-volume-id>/<vmtools-repository-name>
# chmod 700 /vmfs/volumes/<datastore-name-or-volume-id>/<vmtools-repository-name>

This datastore location is assumed to be present and configured as above later.
Find the current ProductLocker location:
From ESXi shell:
# ls -l /productLocker
# readlink /productLocker

The output of the 2^nd command is the absolute path to the current VMware Tools ISO image repository. The 1^st command is there for diagnosis and can be of use later.

Updating and configuring the VMware Tools repository on a datastore

This assumes the VMware Tools repository location on a datastore exists and was created and configured as described in the “before you begin” section above.

1. If the VMware Tools repository has content, it is recommended to empty it rather than overwriting existing files.

#  rm -rf /vmfs/volumes/<datastore-name-or-volume-id>/<vmtools-repository-name>/*

2. Download and extract the VMware Tools packages from Broadcom Support

Note: VMware Tools packages on Broadcom Support contain metadata files and signatures along with ISO images files must be completely extracted without missing files. Missing files during extraction might lead to unexpected errors during VMware Tools install, upgrade, and version status reporting.

3. Update the VMware Tools repository content (pick one of):
   1. Copy the “vmtools” and “floppies” sub-directories from the extracted directory to the VMware Tools repository.

# cp -r /path-to-package-extracted-dir/vmtools /vmfs/volumes/<datastore-name-or-volume-id>/vmtools-repository-name>/
# cp -r /path-to-package-extracted-dir/floppies /vmfs/volumes/<datastore-name-or-volume-id>/vmtools-repository-name>/
 

2. Selectively update the VMware Tools ISO images files. To provide VMware Tools to Windows VMs take the following files from the (latest) VMware-Tools-windows zip and put them into the respective folders "vmtools" and "floppies":

 

vmtools/isoimages_manifest.txt
vmtools/isoimages_manifest.txt.sig
vmtools/windows.iso
vmtools/windows.iso.sha
vmtools/windows.iso.sig
vmtools/windows_avr_manifest.txt
vmtools/windows_avr_manifest.txt.sig
floppies/pvscsi-Windows8.flp
floppies/pvscsi-Windows2008.flp
floppies/pvscsi-WindowsVista.flp

If you want to provide VMware Tools for legacy Linux guest operating systems as well, additionally take the following files from the (latest) VMware-Tools-core zip and put them into the "vmtools" folder.

vmtools/linux.iso
vmtools/linux.iso.sha
vmtools/linux.iso.sig
vmtools/linux_avr_manifest.txt
vmtools/linux_avr_manifest.txt.sig

4. Set the required permissions for the files:

# chmod -R 700 /vmfs/volumes/<datastore-name-or-volume-id>/vmtools-repository-name>/*

Note: All steps hereon (step 5 and following) are for first time setup of the vmtools repository on a shared datastore or when the location of the vmtools repository changes.

5. Configure ProductLocker to use the new VMware Tools ISO images repository location.To update the UserVars.ProductLockerLocation variable to point to the VMware Tools ISO images repository location either use the ESXi shell or the vSphere Web Client:
   1. From ESXi shell:

# esxcli system settings advanced set -o /UserVars/ProductLockerLocation -s  /vmfs/volumes/<datastore-name-or-volume-id>/vmtools-repository-name>

2. From vSphere Web Client:
   1. Select the host and select Configure tab.
   2. Click Advanced System Settings under System.
   3. Click Edit and filter for ProductLockerLocation.
   4. Set UserVars.ProductLockerLocation to /vmfs/volumes/<datastore-name-or-volume-id>/vmtools-repository-name>.
   5. Save the changes.

 

6. Update the ProductLocker symbolic link (/productLocker)​​​​​​​
   1. Reboot the VMware ESXi host, or 
   2. Follow the reboot-less procedure in “Updating the ProductLocker symbolic link without rebooting the VMware ESXi hosts” section below that matches the VMware ESXi Host version.

Note: Using this approach is not recommended for ESXi hosts that are provisioned using Auto Deploy. For more information on configuring host profile for hosts provisioned with Auto Deploy, see VMware Tools for hosts provisioned with Auto Deploy (2004018).

Updating the ProductLocker symbolic link without rebooting the VMware ESXi hosts

After updating the ProductLocker location (UserVars.ProductLockerLocation); the /productLocker symbolic link needs to be updated to the same location. This is usually performed automatically as the VMware ESXi host starts up (reboot or power on). It can also be done manually following one of the procedures herein.

vSphere 6.7 Update 1 or later
In vSphere 6.7 Update 1, updateProductLockerPolicy tool has been deprecated by the updateProductLockerLocation vSphere API, for more information, see UpdateProductLocker in Configuring a VMware Tools Repository in vSphere 6.7U1
Example:
Invoking updateProductLockerLocation vSphere API with /vmfs/volumes/<datastore-name-or-volume-id>/vmtools-repository-name> as the argument, updates the /productLocker symlink

To invoke the API, you can use the vCenter Managed Object Browser. You can access the MOB at https://vcenter_fqdn/mob (replace vcenter_fqdn with actual FQDN or IP of your vCenter Server). Login with the [email protected] account, then browse to [content > rootFolder > childEntity > hostFolder]. Follow the path under childEntity until you arrive at the host.

Alternatively you can open the API directly using the Host ID. For this, select the host in vSphere Client. The URL will include the ID as shown in the picture below:

Use this ID to open the following URL:
https://vcenter_fqdn/mob/?moid=<ID>&method=updateProductLockerLocation
 

In the example this would be:
https://vcenter_fqdn/mob/?moid=host-18022&method=updateProductLockerLocation

Paste the path to the new directory in "Path", then click on "Invoke Method" to apply the setting:

vSphere 6.5 Update 1 or later

For the hosts running ESXi 6.5 Update 1 or later, either use maintenance mode as described above for ESXi 6.5 or use a new tool 'updateProductLockerPolicy'.

1. Get the existing /productLocker target (for step 3 below)

# secpolicytools -d | grep $(basename $(readlink /productLocker)) | cut -d' ' -f2 | head -n1

2. Rename the existing /usr/lib/vmware/isoimages symlink.

# mv /usr/lib/vmware/isoimages /usr/lib/vmware/isoimages.tmp

3. Update the security policy on the host.

# updateProductLockerPolicy <old-symlink-target-retrieved-from-step1> /vmfs/volumes/<volumeName>/<extracted directory>

4. Remove the existing /productLocker symlink.

# rm /productLocker

5. Create a new /productLocker symlink.

# ln -s /vmfs/volumes/<volumeName>/<extracted directory> /productLocker

6. Restore the /usr/lib/vmware/isoimages symlink

# mv /usr/lib/vmware/isoimages.tmp /usr/lib/vmware/isoimages

vSphere 6.5 pre-Update 1

For the hosts running ESXi 6.5 or later but with patches earlier than ESXi 6.5 Update 1, maintenance mode is required to avoid failure of VMs due to the host security policy restricting VMs from suddenly accessing the new /productLocker symlink.

1. Enter the host in maintenance mode

# esxcli system maintenanceMode set --enable true

2. Remove the existing symlink

# rm /productLocker

3. Create the new symlink

# ln -s /vmfs/volumes/<volumeName>/<extracted directory> /productLocker

4. Exit the host from maintenance mode

# esxcli system maintenanceMode set --enable false

vSphere 6.0, 5.x, 4.x

For the hosts running ESXi versions earlier than 6.5, remove the existing symlink and create new one.

1. Remove the existing symlink

# rm /productLocker

2. Create the new symlink

# ln -s /vmfs/volumes/<volumeName>/<extracted directory> /productLocker

After performing the preceding operations, the ESXi host is ready for installation or upgrade of VMware Tools in the virtual machines. In vSphere Web client, right click on the running Virtual Machine object and choose Install or Upgrade VMware Tools option. To install or upgrade VMware Tools, see Installing and upgrading VMware Tools in vSphere

Note:
Currently running VMs on the host will need to be vMotion’d to a different host and back, or shutdown (reset will not work) and powered on again for the updated VMware Tools repository to be accessible.

After rebooting the VMware ESXi host, the productLocker location is not added to the security policies for the VMs.

Following VMware ESXi host reboot, the entry for the productLocker in the VMware ESXi security policies will be updated to the new path. This update can fail if the datastore becomes accessible later than expected in the VMware ESXi host boot process. As a result, VMs security policies do not have the updated productLocker location and the VMware Tools repository is not accessible to the running VMs.

To detect the issue, Inspect the security policies:

# secpolicytools -d | grep $(basename $(readlink /productLocker)) | cut -d' ' -f2 | head -n1

This should return an entry matching the configured path to VMware Tools repository in the form:

-r <absolute-path-to-vmtoolsRepo-on-datastore> r

If not, you are experiencing this issue.
The workaround here is to reload the security policies:

1. Make sure the VMware Tools repository datastore is accessible.
2. Update the security policies manually

# secpolicytools -p

3.  Any running VM will need to be vMotion’d to a different host and back, or shutdown (reset will not work) and then powered on for the change to take effect. VM powered on after the security policies are updated will have the correct VMware Tools repository access. ​​​​​​​

Note: The datastore being slow to become accessible during VMware ESXi host reboot should be investigated, it could be linked to a storage or other resource issues in the environment.