VIrtual Machine performance is degarded when sched.scsi0:0.throughputCap is present

Products

• VMware vSphere ESXi

Issue/Introduction

Virtual Machine (VM) performance is significantly degraded, exhibiting general sluggishness, prolonged boot times, and noticeable latency for all in-guest actions.

This issue often occurs with vCloud Director-managed VMs, particularly after they have been reverted from a snapshot or when deployed from a template associated with a storage policy that defines disk throughput limits.

Environment

VMware vCenter Server 8.x
VMware Cloud Director 10.x

Cause

The performance degradation is caused by an explicit, low disk throughput limit being enforced on the VM's virtual hard disk via the .vmx configuration file.

1. Storage Policy Configuration: Within vCloud Director, the assigned Storage Policy had an IOPS or Throughput Limit defined with a specific, low value (e.g., set to an equivalent of 16 MB/s).

2. VMX Translation: VCD and vSphere translate this storage policy limit into an advanced configuration parameter in the VM's .vmx file:

   sched.scsi0:0.throughputCap = "16"
   

   (Note: This parameter enforces a limit in Megabytes per second (MB/s), not IOPS, where scsi0:0 denotes the specific virtual disk.)

3. Performance Impact: This hard cap dramatically restricts the maximum sequential and random disk throughput available to the VM, crippling performance for disk-intensive operations like operating system boot, application loading, and snapshot consolidation.

Resolution

The resolution involves removing the throughput limit from the source Storage Policy in VCD and then ensuring all affected VMs adopt the corrected setting.

1. Modify the VCD Storage Policy (Prevention)

To ensure all future deployments and reconfigurations are correct, modify the source policy:

• Navigate to the relevant Storage Policy within vCloud Director.

• Edit the policy configuration and set the IOPS or Throughput Limit to 0 (zero).

  • Crucial Note: A value of 0 signifies unlimited or no limit enforced by vSphere, which is the desired state for unconstrained performance.

2. Apply Resolution to Existing VMs (Remediation)

For VMs currently suffering from the issue, you have two methods to clear the constraint:

A. Recommended (Via VCD/vSphere UI):

• Option 1: Storage Policy Change: Change the affected VM's storage policy to the newly modified policy (limit set to 0). Even a quick change and change-back often forces vSphere to re-evaluate and apply the correct "no limit" setting.

• Option 2: Direct VM Edit: In vCenter, edit the VM's settings, navigate to the Virtual Hardware tab, expand the virtual hard disk, and verify that the Disk IOPS limit is set to Unlimited or 0.

B. Direct VMX File Modification (Advanced/Emergency Fix):

• Power off the affected VM.

• Access the ESXi host or datastore browser to locate the VM's .vmx file.

• Edit the .vmx file and modify the throughput line:

  sched.scsi0:0.throughputCap = "16"
  

  Change it to:

  sched.scsi0:0.throughputCap = "off"
  

• Reload the VM in vCenter (or register it if you unregistered it to edit the VMX) and power it on. The VM should now boot without the performance constraint.