After upgrading to ESXI 6.7 U1, VMs fails to power on with Error message: PCI passthrough devices cannot be added when Nested Hardware-Assisted Virtualization is enabled.

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• VDI desktop VMs with Graphics cards attached as Passthrough device fails to power on after upgrading to ESXi 6.7 Update 1
• VM power on Fails with error message similar to:

PCI passthrough devices cannot be added when Nested Hardware-Assisted Virtualization is enabled.

• If you remove the PCI Passthrough device you cannot re-add it back.

Environment

VMware vSphere ESXi 7.0.x
VMware vSphere ESXi 6.7

Cause

By Design, PCI passthrough and nested hardware assisted virtualization are not enabled simultaneously. However a configuration check was not enforced in previous versions. Additional validation was added in hostd from ESXI 6.7 Update 1 to ensure PCI passthrough and nested hardware assisted virtualization are not enabled simultaneously  (vGPU and VVTD):

Resolution

It is not recommended to enable both nested hardware assisted virtualization and PCI Passthough

Workaround:
Disable hardware assisted virtualization to the guest OS