VMFS extent Offline, causing VMs to go unresponsive

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms: 

The esx.problem.vmfs.extent.offline message is received when an ESXi host loses connection to a storage device that backs an entire VMFS datastore or any of its extents.

 

Impact

I/O to virtual disk (VMDK) files that reside on the offline device might slow down or fail. This might cause virtual machines that have VMDKs residing on that device to become unresponsive or even fail.

 

Example

This loss of connection can happen when a switch or cable that connects the device to the ESXi host is disconnected or when the device is reformatted to be used by another volume.

Resolution

Detect the problem with device connectivity and restore connectivity to the device. If the device has been reformatted and used by another volume, then this part of the original volume is lost forever and cannot be recovered.

 

To determine which devices back a vmfs volume, and if one or more devices may be offline, run:

vmkfstools -Ph /vmfs/volumes/<datastore>

Example: 

 

[root@host1:~] vmkfstools -Ph /vmfs/volumes/datastore
VMFS-6.82 (Raw Major Version: 24) file system spanning 1 partitions.
File system label (if any): datastore
Mode: public
Capacity 399.8 GB, 156.2 GB available, file block size 1 MB, max supported file size 64 TB
Disk Block Size: 512/16384/0
UUID: 66####ac-11####88-7##1-00#######257
Partitions spanned (on "lvm"):
        naa.6589#########################758:1
Is Native Snapshot Capable: NO

 

Note: If the datastore is configured with multiple extents, under the ‘Partitions spanned (on “lvm”)’ head, you’ll find all the associated naa.ids listed