Spread VMs across GPUs option does not take affect when using mixed mode profile

Products

• VMware vSphere ESXi

Issue/Introduction

• You have multiple GPUs on ESXi host and want to use 'Spread VMs across GPUs' policy for spreading GPU workload on multiple VMs.
  
  
• However, when using Mixed Size GPU mode or MIG mode, VMs are not evenly assigned to the multiple GPUs when powered on.
  
  
• When VMs are powered on, they are not evenly assigned to multiple GPUs.
  
  
• You can verify this in the host log bundle within the /command/vmware-vimdump_-o----U-dcui.txt file. 
  As shown below, most VMs are assigned to the exact same PCI ID (e.g., 0000:XX:00.0):
  

                 graphicsInfo = (vim.host.GraphicsInfo) [
                    (vim.host.GraphicsInfo) {
                       dynamicType = <unset>,
                       dynamicProperty = (vmodl.DynamicProperty) [],
                       deviceName = 'NVIDIA H100',
                       vendorName = 'NVIDIA Corporation',
                       pciId = '0000:XX:00.0',
                       graphicsType = 'sharedDirect',
                       vgpuMode = 'mixedSize',
                       memorySizeInKB = ########,
                       vm = (vim.VirtualMachine) [
                          'vim.VirtualMachine:##',
                          'vim.VirtualMachine:##',
                          'vim.VirtualMachine:##',
                          'vim.VirtualMachine:##',
                          'vim.VirtualMachine:##',
                          'vim.VirtualMachine:##',
                          'vim.VirtualMachine:##',
                          'vim.VirtualMachine:##',
                       ]
                    },
                    (vim.host.GraphicsInfo) {
                       dynamicType = <unset>,
                       dynamicProperty = (vmodl.DynamicProperty) [],
                       deviceName = 'NVIDIA H100',
                       vendorName = 'NVIDIA Corporation',
                       pciId = '0000:ZZ:00.0',
                       graphicsType = 'sharedDirect',
                       vgpuMode = 'mixedSize',
                       memorySizeInKB = ########,
                       vm = (vim.VirtualMachine) [
                          'vim.VirtualMachine:##',
                       ]

 

Environment

VMware vSphere 8.x

Cause

The "spread VMs across GPUs" policy applies exclusively when a system has multiple devices configured in vGPU time-sliced sameSize mode. 

It does not apply to devices in mixedSize or MIG mode. 

This design is necessary to minimize hardware fragmentation during placement by the Assignable Hardware component.

Resolution

1. Acknowledge that this is expected behavior when vGPU is configured in mixedSize or MIG mode.

2. Note that the ESXi host configuration "Spread VMs across GPUs" is intentionally bypassed and not used when the device is in mixedSize or MIG mode.

3. To successfully utilize the "Spread VMs across GPUs" option, you must configure the environment to use time-sliced Same-Size vGPU Mode.