Duplicate Dynamic Disks show up as Missing in Windows Disk Manager after removing a snapshot

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• When you create an application-consistent quiescent snapshot of a virtual machine running Windows Vista, 7, 2003, or 2008, you see duplicate dynamic disks in the Windows Disk Manager.
• When removing the virtual machine snapshot, the duplicate dynamic disks appear as Missing in the Windows Disk Manager.
• In some cases, after the virtual machine snapshot removal, the drive letters previously assigned to the original dynamic disk volumes become unassigned.
• The file system for the respective dynamic volumes may remount only in read-only mode.

Environment

VMware ESXi 4.0.x Embedded
VMware ESX 4.1.x
VMware ESXi 4.1.x Installable
VMware ESX 4.0.x
VMware ESXi 4.1.x Embedded
VMware ESXi 4.0.x Installable

Cause

Hot-Add functionality is not intended for use with Dynamic Disks. In some versions of VMware ESX/ESXi 4.0 and 4.1, detection of dynamic volumes may be unsuccessful, which may result in the Hot-Add mechanism inadvertently being used.

Depending on guest operating system configuration, Hot-Add actions may result in not retaining drive letters for all attached dynamic disk volumes due to DiskPart being configured (by default) with automatic drive letter assignment.

Further, the remaining dynamic volumes remain aware of their now-missing peers (or snapshot-copies), resulting in interpreting them as Missing in the Disk Manager.

Resolution

To correct automatic fallback to File System quiescing behavior and prevent Hot-Add functionality from being used for these specific virtual machines, update VMware ESX/ESXi 4.1 to Update 2 or upgrade to VMware ESXi 5.0. To download, visit the VMware Download Center.

To work around the issue, perform one of these options:

• Power off the virtual machine, add this to the VMX configuration file, then restart the virtual machine:
  
  disk.EnableUUID = "FALSE"
  
  
• Via VMware Tools inside the guest operating system while it is running:
• 1. Open C:\ProgramData\VMware\VMware Tools\Tools.conf.
     
     Note: If this file does not exist, create it.
     
     
  2. Add these lines to the file:
     
     [vmbackup]
     vss.disableAppQuiescing = true
     
     
  3. Save and exit the file.
  4. Restart the VMware Tools service for the change to take effect.

Note: If EMC Replication Manager is being used to create virtual machine snapshots, you may set a global environment variable to disable application-consistent quiescing. Contact EMC for further instructions.

Additional Information

Hot-add is used when performing application-consistent quiescent snapshots on a Windows 2003+ virtual machines with dynamic disks. It utilizes VSS (Volume Shadow Copy Service); Hot-add is used to provide writer space for a VSS component.

All attached disks are included in the VM snapshot and are re-attached to the system a second time. The disks show up multiple times in the disk manager simultaneously.

Consider this example:

1. A Windows 2008 R2 virtual machine has two disks. One is Basic (boot), and the other is Dynamic.
2. The virtual machine is snapshotted with the quiesce option.
3. Unlike before, this virtual machine now has four delta files. Two are for the disks' ongoing delta/change records, post-snapshot, and the other two serve as writeable space for VSS components in the Guest.
4. In the Disk Manager, you now see four disks, as well. Two identical operating system disks (both basic), and two identical Dynamic disks.
5. • In this process, the dynamic disks actually have now registered the existence of other dynamic volumes found in the system. They have added each other to their logical disk management (LDM) tables.
     
     
   • If you check the virtual machine, you see Disk 0 is the operating system (on a delta disk now), Disk 1 is the dynamic disk. Disk 2 is the duplicate of Disk 0 (operating system), which is used for the VSS writer delta. Disk 3 is the duplicate of the dynamic disk, Disk 1.
     
     
6. When you remove/delete the snapshot finally (such as when your backup is done), it consolidates the normal running deltas and discards the VSS writer deltas. It does not matter what is consolidated or discarded in this step. The virtual machine snapshot is removed and is now back to running on the base disks.
7. Unfortunately the guest operating system's logical disk manager at this stage sees that a dynamic disk is now missing. It is now a Missing disk, which was used only as temporary VSS writer space.
8. The other symptoms described earlier in this article arise from the system interpreting it no longer has access to a required disk for a given volume set (hence even the normal disk, Disk 1, showing up as failed/missing, while it is definitely attached.)