Snapshot using a Snapshot Manager fails with the error: "Operation not allowed in current state"

Products

• VMware vCenter Server 6.0
• VMware vCenter Server 7.0
• VMware vCenter Server 8.0
• VMware vSphere ESXi 6.0
• VMware vSphere ESXi 7.0
• VMware vSphere ESXi 8.0

Issue/Introduction

• Attempting a snapshot for a virtual machine using Snapshot Manager, or VMware Consolidated Backup or a third-party backup solution such as vRanger fails.
• You see these errors in vCenter Server:
  
  Cannot create snapshot
Operation not allowed in current state
   
• The virtual machine, VMware vCenter Server, and the host appear to be running without problems, but the error persists.
• ESXi - /var/log/hostd.log contains entries similar to:
  
  [YYYY-MM-DDTHH:MM:SS.Z error 'vm:/vmfs/volumes/######-######-###-######/VMNAME/VMNAME.vmx'] Invalid transition requested (VM_STATE_ON_SHUTTING_DOWN -> VM_STATE_CREATE_SNAPSHOT): Invalid state

Environment

• vSphere 6.x
• vSphere 7.x
• vSphere 8.x

Cause

This issue occurs if the vCenter Server management agents have stopped causing the snapshot attempt to move into the ON_SHUTTING_DOWN state.

Resolution

Restart the VMware agents on the host.

1. Ensure that the Virtual machine startup / shutdown with host feature is disabled before this step. 
2. Restart the management agents on the host. For more information, see Restarting the Management agents in ESXi
3. On vCenter Server, restart the VMware Virtual Center service. For more information, see How to stop, start, or restart vCenter Server services
4. Attempt the snapshot again.
5. The operation succeeds and the corresponding log entry for the snapshot attempt reflects a transition from ON to CREATE SNAPSHOT instead of ON_SHUTTING_DOWN.

Workaround

• This issue can occur when the PCI device is attached to virtual machine. Snapshots are not supported with the PCI devices.  
• Remove the PCI device to create the snapshot or power off the virtual machine and take a snapshot.