Virtual machines with NVIDIA GPU profiles take a long time to power off

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

• Virtual Machine power off operations takes up to a few minutes when large amounts of pass thru guest memory is assigned to the virtual machine.
• Both vDGA and NVIDIA vGPU configurations are affected.
• The amount of time it takes to power off a virtual machine is directly proportional to the amount of memory assigned to the machine.

Environment

• VMware vSphere ESXi 6.0
• VMware vCenter Server 6.0.x

Cause

The Virtual Machine power off latency is caused by a large amount of IOMMU IOTLB flush operations.

The same power off slowdown occurs with other passthru virtual machines where the slowdown is proportional to guest memory size.

Resolution

Resolved in ESXi 6.0 u2 and later.