High frequency of read operations on VMware Tools image may cause SD card corruption

Products

• VMware vSphere ESXi

Issue/Introduction

The purpose of this article is to highlight a recent optimization made to ESXi to reduce read wear on its bootable SD card.

Symptoms:

• ESXi host fails to boot while loading the boot image from an SD card
• Running an fsck check against the bootable SD card returns errors

Environment

• VMware vSphere ESXi 6.0
• VMware vSphere ESXi 7.0.0
• VMware vSphere ESXi 6.5
• VMware vSphere ESXi 6.7

Cause

SD cards are not created equal. Each SD card make/model is designed to tolerate different numbers of read/write IO operations. VDI environments frequently issue read operations to the SD card to access VMware Tools. The frequency or these read operations can cause SD cards to burn out prematurely resulting in SD card corruption.

Resolution

• This issue is resolved in ESXi 6.0 Update 3.
• For ESXi 6.5, the issue is resolved in Update 1.
• For ESXi 7.0, the issue is resolved in Update 3c
• In ESXi 7.0 Update 3 and later, the locker packages are loaded into RAM disk by default if they are stored on a USB or SD card device. The activation steps below are applicable to earlier versions of ESXi.
• In ESXi 6.0 Update 3 and later, changes were made to reduce the number of read operations being sent to the SD card, an advanced parameter was introduced that allows you to migrate your VMware tools image to ramdisk on boot . This way, the information is read only once from the SD card per boot cycle.

To activate this feature:

Note: This process requires ESXi host reboots.
 

1. Log in to each host using an SSH connection and set the ToolsRamdisk option to 1:
   
   esxcli system settings advanced set -o /UserVars/ToolsRamdisk -i 1
    

Note: If the ToolsRamdisk option is not present, add the option using the following command:

esxcfg-advcfg -A ToolsRamdisk --add-desc "Use VMware Tools repository from /tools ramdisk" --add-default "0" --add-type 'int' --add-min "0" --add-max "1"

 

2. Reboot the ESXi host
3. Repeat for remaining hosts in the cluster.
   To check the current value of "ToolRamdisk", run the following command
   
   # esxcli system settings advanced list -o /UserVars/ToolsRamdisk