VMs on SmartNIC Hosts Fail to Start with a vCenter power-on VM Task Status NIOC Message: Network I/O admission control (NIOC) check failed for Ethernet device(s)

Products

• VMware vSphere ESXi 8.0

Issue/Introduction

• VMs fail to start and vCenter power-on VM task status messages similar to the below are observed:
  

  Network I/O admission control (NIOC) check failed for Ethernet device(s): 1(bitMap). 
  Check the host network configuration for available bandwidth for reservation. 
  Failed to start the virtual machine. 
  Module NIOC power on failed.

• Impacted VMs are utiliizing host SmartNICs.  SmartNIC driver details can be displayed by using the below CLI command from the VM host:
  
  [root@esx:~] esxcli network nic list

Environment

VMware vCenter Server

VMware vSphere ESXi

Cause

There are two potential causes for a VM power-on failure due to NIOC.  This KB is specific to 'Cause 2' below:

Cause 1
ENS-Polling mode does not support more than 108 VMs per vSwitch:

• See the following KB for further details:  
  VMs connected to a vSwitch in ENS Polling Mode are intermittently failing to start and vCenter power on VM task status NIOC messages are observed:  Network I/O admission control (NIOC) check failed for Ethernet device(s)

Cause 2
NIOC is not supported by smartNICs.

 

Resolution

This is a known issue impacting VMware NSX.

 

Cause 2 Workaround

1. Remove ethernetX.share, ethernetX.reservation and ethernetX.limit from the .vmx file of impacted VMs:

• To confirm if these settings are present on the impacted VMs run the following CLI command from the VM host:
  grep -i 'ethernet' /vmfs/volumes/<Storage Directory>/<VM Name>.vmx