Taking a quiesce snapshot of Windows Server 2019 virtual machine fails

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Taking a quiesce snapshot of Windows Server 2019 virtual machine fails.
• You get an error similar to:
  
  An error occurred while taking a snapshot: Failed to quiesce the virtual machine.

• In the vmtools vss logging, you see entries similar to:
  
  2019-07-02T11:32:11.050Z| vmx| I125: Guest: [  debug] [vmvss:vmvss] CVmSnapshotRequestor::NeedWaitForSnapshotCompleted():1082: enter
  2019-07-02T11:32:11.190Z| vmx| I125: Guest: [  debug] [vmvss:vmvss] CVmSnapshotProvider::OnLunEmpty():716: 0x000002BB00A65C30
  2019-07-02T11:32:11.190Z| vmx| I125: Guest: [  debug] [vmvss:vmvss] CVmSnapshotProvider::OnLunEmpty():716: 0x000002BB00A65C30
  2019-07-02T11:32:11.205Z| vmx| I125: Guest: [  debug] [vmvss:vmvss] CVmSnapshotProvider::AbortSnapshots():677: 0x000002BB00A65C30
  2019-07-02T11:32:11.216Z| vmx| I125: Guest: [  info] [vmvss:vmvss] CVmSnapshotRequestor::WaitForOperation():3364: Unexpected esult from QueryStatus(): 0x800423f9
  2019-07-02T11:32:11.217Z| vmx| I125: Guest: [ warning] [vmvss:vmvss] CVmSnapshotRequestor::WaitForOperation():3373: Async opeation result: IDispatch error #8697
  2019-07-02T11:32:11.218Z| vmx| I125: Guest: [  debug] [vmvss:vmvss] CVmSnapshotRequestor::DoSnapshotSet():1904: failed call: et = WaitForOperation(async), result = 0x800423f9
  2019-07-02T11:32:11.218Z| vmx| I125: Guest: [  debug] [vmvss:vmvss] CVmSnapshotRequestor::Finalize():2616: enter
  2019-07-02T11:32:11.242Z| vmx| I125: Guest: [  debug] [vmvss:vmvss] CVmSnapshotRequestor::UnmountSnapshotVolumes():2439: enter
  2019-07-02T11:32:11.242Z| vmx| I125: Guest: [  debug] [vmvss:vmvss] VDSHelper::ForEachVolume():206: enter
   
• There is error log in the Application Log of Windows Events.

An expected hidden volume arrival did not complete because this LUN was not detected.

Environment

VMware vSphere ESXi 8.0
VMware vSphere ESXi 7.0
VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.5

Cause

This issue occurs when Windows is unable to identify the recovery volume that is included in the VSS SnapshotSet during the VSS process.

The issue only happens when the Windows 2019 VM is EFI-boot, has the recovery volume and the system disk is GPT disk. It is identified as Microsoft VSS bug.

Resolution

This issue is resolved in Windows Server version 1809 and all versions of Windows Server 2019 OS Build 17763.1012. For more information, see January 23, 2020—KB4534321 (OS Build 17763.1012).

Disclaimer: VMware is not responsible for the reliability of any data, opinions, advice, or statements made on third-party websites. Inclusion of such links does not imply that VMware endorses, recommends, or accepts any responsibility for the content of such sites.

Workaround:
There are 2 options to work around this issue:

Option 1 - Use BIOS boot and MBR disks since Windows 2019 installation

For EFI systems, Microsoft requires Windows should be installed on GPT disks. So it’s suggested to resolve this issue by using an MBR disk layout instead of GPT while provisioning the machines.

Note: MBR can be used if the disk is <= 2TB. If a machine already has GPT, backup all the data from the disk, and convert the disk to MBR . For more information, see the Microsoft article Convert a GPT disk into an MBR disk.

Option 2 - Fall back to the filesystem-consistent snapshot

1. Navigate to %SystemDrive%\ProgramData\VMware\VMware tools.
2. Create/edit the Tools.conf file with a text editor.
3. Add this entry (Or modify existing entry as required).
   [vmbackup]
   vss.disableAppQuiescing = true
4. Save the file.