Error: "Protocol error from VMX" when taking a Windows vm snapshot

Products

• VMware vSphere ESXi

Issue/Introduction

• Snapshot of a Windows server that has the Quiesced Guest file system option enabled fails
• Cloning a virtual machine fails
• Application-consistent quiescing has been enabled in Windows guest
• You see the error:
  
  A general system error occured: Protocol error from VMX.
   
• In the vmware.log file, you see the error:
  
  Toolsbackup: not enough empty nodes (needed <number of attached disks>, found <number of disks>).
  
  For example: vmx| ToolsBackup: not enough empty nodes (needed 8, found 7)
  
  Where <number of attached disks> is the number of disks attached to the virtual machine. 
• This may affect any version of Windows.

Environment

• VMware vSphere ESXi 6.0
• VMware vSphere ESXi 6.5
• VMware vSphere ESXi 7.0.0
• VMware vSphere ESXi 5.5

Cause

The issue occurs when the Windows server has Application-consistent quiescing enabled and the virtual machine has more than 7 disks attached to one SCSI controller.

Resolution

Use one of these options:

• Add an additional controller to the virtual machine, so that there are sufficient empty disk nodes on the controllers for the snapshots.
  
  To add an additional controller when the virtual machine is running:
  1. Add a new disk and in the Advance Options step.
  2. Change the Virtual Device Node to use another controller.
     
     For example, use SCSI 1 instead of SCSI 0.
     
     When the virtual machine is down, you can add another controller without adding a new disk. To add another controller,:
     1. Edit the settings of one of the existing disk
     2. Change Virtual Device Node to use a different controller.
        
        For more information on adding an additional controller, see the vSphere Virtual Machine Administration Guide.
         
• Disable application-consistent quiescing on guest OS. Set disk.EnableUUID = "FALSE" in the virtual machine configuration file. For more information, see Enabling and disabling Windows 2008 and later application-consistent quiescing on ESXi/ESX.