Virtual machine snapshot fails with "operation is not allowed in the current state" when PCI passthrough device is present

Products

• VMware vCenter Server

Issue/Introduction

• You attempt to take a snapshot of a virtual machine and the task fails with an error similar to:

  Operation failed!
  Task name    Create virtual machine snapshot
  Status       The operation is not allowed in the current state.
  

• In the \var\log\hostd.log on the ESXi host, you see errors similar to:

  YYYY-MM-DDTHH:MM:SS.SSSZ Db(167) Hostd[#####]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/<datastore>/<vm-folder>/<vm-name>.vmx opID=<hex-value> sid=<hex> user=vpxuser:<domain>\<username>] Create Snapshot: VM Snapshot, memory=false, quiescent=false state=5
  YYYY-MM-DDTHH:MM:SS.SSSZ Wa(164) Hostd[#####]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/<datastore>/<vm-folder>/<vm-name>.vmx opID=<hex-value> sid=<hex> user=vpxuser:<domain>\<username>] State Transition (VM_STATE_ON -> VM_STATE_CREATE_SNAPSHOT) not allowed for this Vm
  YYYY-MM-DDTHH:MM:SS.SSSZ In(166) Hostd[#####]: [Originator@6876 sub=AdapterServer opID=<hex-value> sid=<hex> user=vpxuser:<domain>\<username>] AdapterServer caught exception: vim.VirtualMachine.createSnapshot, N3Vim5Fault12InvalidState9ExceptionE(Fault cause: vim.fault.InvalidState
  

• The virtual machine is powered on and has a PCI or PCIe passthrough (DirectPath IO) device attached.
• Backup solutions that rely on snapshots (such as Veeam) also fail for this virtual machine.

Additional symptoms reported:

• Unable to take the snapshot on one of the VMs.

Environment

• ESXi 8.0
• vCenter Server 8.0
• Virtual machine with a PCI/PCIe passthrough (DirectPath IO) device

Cause

VMware does not support snapshots on virtual machines that have PCI or PCIe passthrough (DirectPath IO) devices attached. When a physical device is passed through to a VM, the hypervisor cannot capture or freeze the device state as part of the snapshot process. The snapshot task is blocked and ESXi returns an InvalidState fault.

The VM's .vmx configuration file shows the passthrough device. For example:

pciPassthru0.present = "TRUE"
pciPassthru0.deviceId = "0x101f"
pciPassthru0.vendorId = "0x15b3"

The vSphere Client also displays a note on the PCI device entry: "Some virtual machine operations are unavailable when PCI/PCIe passthrough devices are present."

Resolution

To take a snapshot, you must first remove the PCI passthrough device from the virtual machine.

1. Open the vSphere Client and locate the affected virtual machine.
2. Power off the virtual machine.
3. Right-click the virtual machine and select Edit Settings.
4. Locate the PCI device entry in the hardware list.
5. Click the X (remove) button next to the PCI device to remove it.
6. Click OK to save the changes.
7. Power on the virtual machine.
8. Take the snapshot.

If you need to keep the PCI passthrough device:

1. Power off the virtual machine.
2. Take the snapshot while the VM is powered off. Snapshots succeed on powered-off VMs even with passthrough devices present.
3. Power the virtual machine back on.

Additional Information

• For more information on PCI passthrough limitations, see ESXi pci passthrough device limits | VMware vSphere.
• For a related snapshot failure scenario, see Snapshot using a Snapshot Manager fails with the error: "Operation not allowed in current state".