Virtual machine stops responding with the error: The lock protecting virtualdisk.vmdk has been lost

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

A virtual machine that is in a powered-on state and running goes into an inconsistent state suddenly.
You see a warning similar to:

The lock protecting virtualdisk.vmdk has been lost. This is most likely due to underlying storage having problems, resulting in this virtual machine getting powered on at another ESX host as well. This virtual machine needs to be powered off at this host now. Kindly confirm that the virtual machine is running successfully on another host before clicking the OK button

or

The lock protecting test.vmdk has been lost, possibly due to underlying storage issues. If this virtual machine is configured to be highly available, ensure that the virtual machine is running on some other host before clicking OK
When you click OK, the virtual machine shuts down.
This issue is observed on ESX hosts in HA-enabled clusters. It can also occur in non-HA setups when the datastores hosting virtual machines are shared across ESX hosts.
In HA enabled clusters, the virtual machine gets powered-on on another host in the cluster.

Environment

VMware vCenter Server 5.0.x
VMware vSphere ESXi 6.5
VMware ESXi 4.0.x Installable
VMware vCenter Server 4.0.x
VMware ESXi 4.1.x Embedded
VMware vCenter Server 5.5.x
VMware vSphere ESXi 5.5
VMware vCenter Server 5.1.x
VMware ESXi 4.0.x Embedded
VMware vSphere ESXi 5.0
VMware ESXi 4.1.x Installable
VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.0
VMware vCenter Server 4.1.x
VMware vSphere ESXi 5.1

Resolution

If an ESX/ESXi host loses access to a datastore, I/O from running virtual machines on the datastore will time out and fail. The virtual machine pauses and an event message appears stating that the virtual machine lost access to its disk. This problem might occur in these situations:

The host's storage network connection is not restored within 15 seconds and another host breaks the disk lock.

This is the expected behavior because VMFS host clustered lock manager lets a host send I/O to resources such as virtual disk files only if the host owns the resource through a lock. This process is required so that guest data remains consistent with other hosts who might try to access the same data. For more information see, Host Connectivity Degraded in ESX/ESXi (1009557).
The host is part of a VMware HA cluster and loses connectivity to its management and storage networks (Isolation Event).

In this case, VMware HA attempts to restart virtual machines on a 'healthier' host in the cluster. If HA is configured to leave virtual machines powered on when isolated, the virtual machines on the isolated host are failed over, but the original virtual machines remain running on the isolated host (without the VMDKlocks). When the isolated host rejoins the cluster, the duplicate virtual machines running on it fail to reacquire the disk locks and the event message appears.

A node that is isolated from the network needs time to release the virtual machine's VMFS locks if the host's isolation response is to fail over the virtual machines or to leave them powered on.

In HA enabled clusters, the virtual machine will be powered-on on another host. In non-HA clusters, you can power on the virtual machine after it is powered off.

The most common reasons for the failure in updating the disk locks are intermittent SAN and network issues, such as unreachable storage or high latencies. Ensure to resolve any issues with the storage or network in your environment.

For related information, see:

For documentation regarding VMware High Availability and other related features, see:

Additional Information

For translated versions of this article, see:

日本語: 仮想マシンが次のエラーにより応答を停止する： virtualdisk.vmdk を保護するロックが失われました (2101714)