Virtual Machine Fails to Power On with Error: "Module 'PCIPluginLate' power on failed" and "Could not initialize plugin 'libnvidia-vgx.so'"

Products

• VMware vSphere ESXi

Issue/Introduction

• A virtual machine equipped with NVIDIA vGPU fails to power on within a cluster. The following error messages are displayed in the vSphere Client or recorded in the vmware.log:
  

  YYYY-MM-DDThh:mm:ss.xxxZ In(0x)+ vmx - Power on failure messages: Could not initialize plugin 'libnvidia-vgx.so' for vGPU 'nvidia_a16-1b'.
  YYYY-MM-DDThh:mm:ss.xxxZ In(0x)+ vmx - Module 'PCIPluginLate' power on failed.
  YYYY-MM-DDThh:mm:ss.xxxZ In(0x)+ vmx - Failed to start the virtual machine.

• SR-IOV on BIOS / UEFI level enabled.

 

Environment

VMware vSphere ESXi 8.x
VMware vCenter Server 8.x
NVIDIA A16 / A2 vGPU devices

Cause

Specific NVIDIA GPUs, such as the A16 or A2, require SR-IOV (Single Root I/O Virtualization) to be enabled at both the BIOS/UEFI level and within the ESXi host settings to function as a vGPU.
If SR-IOV is disabled for the relevant PCI device on the ESXi host, the vGPU plugin will fail to initialize.

Resolution

Enable SR-IOV for the relevant PCI device on the target ESXi host via the vSphere Client and then restart the ESXi host.

1. Log in to the vSphere Client and select the target ESXi host from the inventory.
2. Navigate to the Configure tab.
3. Click PCI Devices within the "Hardware" section of the left menu.
4. From the list of "All PCI devices," select the target GPU (e.g., NVIDIA A16).
5. Click Configure SR-IOV above the selected device's details.
6. In the pop-up window, configure the following:
   
   • Set the Enable SR-IOV checkbox to On.
   • Enter the supported number of functions in the Number of Virtual Functions field (contact NVIDIA if the required number is unknown).
7. Click OK to save the settings.
8. Restart the target ESXi host to apply the changes to the system.