VMs get corrupted on vVOL datastores after vMotion

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:
When a VM residing on a vVOL datastore is migrated using vMotion to another host by either DRS or manually, and if the VM has one or more of the following features enabled:

• CBT
• VFRC
• IOFilter
• VM Encryption

You see this symptom:

A corruption of data/backups/replicas and/or performance degradation is experienced after vMotion.

Cause

This issue occurs because when a VM with FDS (File Device) filter attached, residing on a vVOL datastore, is migrated using vMotion to another host, the FDS (File Device) filter is not correctly reinserted into the I/O stack at the destination. As a result, vSphere features implemented as a FDS filter viz. CBT, VFRC, IOFilters, VM Encryption will not see the VM I/Os after vMotion.

This would result in corrupted incremental backups (CBT), performance degradation (VFRC and Cache IOFilters), corrupted replicas (Replication IOFilters) and disk corruption (VM Encryption and Cache IOFilters configured in write-back mode).

Resolution

This issue is resolved in:

• ESXi600-201807401-BG and ESXi600-201807001, available at VMware Patch Downloads.
• ESXi650-201811002, available at VMware Patch Downloads.
• ESXi 6.7U1, available at VMware Downloads.

Workaround:
To work around this issue, disable DRS for VMs residing on a vVOL datastore with one of the above features enabled. Whenever manual vMotion of a VM is necessary, perform following steps after vMotion depending on which feature is enabled.

1. CBT - Reset CBT, and do a full backup.
2. VFRC - Disable VFRC and then enable it again.
3. Cache IOFilters configured in write-through mode (also called read mode) - Detach the filter and attach it again.
4. Replication IOFilters - Take a snapshot and initiate a full resync of the replicas.
5. VM Encryption and Cache IOFilters configured in write-back mode - There is no workaround.