Virtual Machine deployment fails with "context deadline exceeded" during Eager Zeroed Thick disk creation

Products

• VMware vCenter Server

Issue/Introduction

• Virtual Machine deployments via Terraform fail intermittently during the disk provisioning phase. The task times out in the automation layer, although the task may eventually complete successfully in vCenter.

Terraform logs report: Error: Post "https://vcenterfqdn/pbm": context deadline exceeded: RESOURCE (vm-<ID>), ACTION (PolicyIDByVirtualMachine)

• In the vCenter, the virtual machine reconfigure task takes longer time to complete
• In the var/log/vmware/vpxd.log on the vCenter, you will see multiple eager zeroed thick disk create tasks

-->    deviceChange = (vim.vm.device.VirtualDeviceSpec) [
-->       (vim.vm.device.VirtualDeviceSpec) {
-->          operation = "add",
-->          fileOperation = "create",
-->          device = (vim.vm.device.VirtualDisk) {

-->                thinProvisioned = false,
-->                eagerlyScrub = true,
-->                sharing = "sharingNone",

• In the var/log/hostd.log on the ESXi host, you will see below entries where disk creation tasks are completed

CreateDisk: Specified disk size is ###### KB

Disk created successfully.

Environment

• VMware vCenter Server
• Terraform vSphere Provider

Cause

• This issue occurs when the timeout in vSphere Terraform provider is configured to a lower value compared to the time taken for the disk creation.
• When deploying VMs with Eager Zeroed Thick (EZT) disks, the storage layer must zero out all blocks at the time of creation. For large disks or multiple disks, this physical I/O operation can exceed the timeout configured in vSphere Terraform provider, causing the task to timeout while vCenter is still processing the request.

Resolution

Option 1: Increase Terraform Resource Timeouts

• Modify the vsphere_virtual_machine resource in Terraform configuration to extend the allowable time for creation and update operations.

Example: 

resource "vsphere_virtual_machine" "vm" {
  # ... other configuration ...

  timeouts {
    create = "20m"
    update = "20m"
    delete = "20m"
  }
}

Option 2: Change Disk Provisioning Policy

If immediate disk zeroing is not required, change the disk type to thin. This allows the reconfiguration task to complete nearly instantaneously.

Example:

Set eagerly_scrub = false and thin_provisioned = true.