How to increase maxIntrCookies if default limit is reached in ESXi ?

Products

• VMware vSphere ESXi

Issue/Introduction

This article will provide a workaround to increase the number of interrupt cookies.

Symptoms:

1. Powering on a virtual machine with PCI Passthrough devices fails:                                                                              

•   In the vmware.log file of the affected virtual machine, you see entries similar to
    ####:##:##.# failed to register interrupt                                      

•   In the vmkernel.log file of the ESXi host where the virtual machine is running, you see
     entries similar to: 
     Unable to find a free interrupt number                    

2. Detection of devices fails while booting ESXi.

•   In the vmkernel.log file of the ESXi host, you see entries similar to: 

    Unable to find a free interrupt number

3. ESXi doesn't detect all the vmnic adapters, But, those vmnic adapters are seen in "lspci", not on esxcfg-nics -l

•   In the vmkernel.log file of the ESXi host, you see entries similar to: 

    ####:##:##.# failed to allocate 64 MSIX interrupts

Environment

• VMware vSphere ESXi 6.x
• VMware vSphere ESXi 7.0.x
• VMware vSphere ESXi 8.0.x

Cause

• This issue occurs due to an ESXi limit on interrupt resources. When PCI Passthrough devices request many interrupts to be allocated, it is possible that this limit has been reached which causes the issue.

Resolution

• The "maxIntrCookies" default value was increased to 4096 in ESXi version 8 to address this issue. Update host "maxIntrCookies" settings to 4096 where values are found to be set below 4096.
• Workaround:
  • To work around this issue, adjust the boot option "maxIntrCookies" to 4096 to increase the interrupt limit.
• The default value for "maxIntrCookies" depends on the ESXi version in use and whether the advanced settings value was manually set higher before a host was upgraded to ESXi version 8 where the default value was increased.  
• ESXi version 7.x "maxIntrCookies" default value is set to 1024
• The default value in ESXi version 8 was increased to 4096.

Note: The maximum interrupt cookies is capped to 4096 even if you set it to a value that is > 4096.
For more information on instructions on how to configure boot options, see Configuring advanced options for ESXi/ESX (310338).